AI Enablement Radar week 47: does AI catch the revised delivery date every time?

AI Enablement Radar week 47: does AI catch the revised delivery date every time?

If AI prepares Monday's supplier meeting, the buyer needs to know whether it spots the revised delivery date every time. This week Google added two measures that distinguish a successful run from a string of successful runs. You can test the same distinction at your desk with an old quote and 40 minutes. Source: Google ADK.

Top signals this week

  • Google's Agent Development Kit for Python can now report whether an agent succeeded at least once in a set of attempts and whether it succeeded on every attempt. Source: Google ADK.
  • Microsoft began rolling out Autopilot to Frontier customers. The agent can monitor a Teams channel and handle recurring follow-ups. Check access before building a routine around it. Source: Copilot Autopilot.
  • Google Drive administrators can preview the labels Gemini would apply to files before putting a new classification rule to work. Source: Google Drive.
  • Apollo Hospitals already uses an agent to flag reported incidents for clinicians at their daily review. Staff decide which cases warrant investigation. Source: Apollo Hospitals.

What organizations are actually doing with AI

At Apollo, the agent summarizes incidents ahead of the meeting. Hospital records show that after rollout, the team closed more than 60 percent of the cases it reviewed within three days. That measures turnaround time. To say how much time the agent saved, you'd also need a comparable figure from before the rollout. Source: Apollo Hospitals.

LVMH's Media Division is at a different stage. Its agreement with OpenAI includes ChatGPT Enterprise and training tailored to each team's work. Journalists will explore document comparison and preliminary translation, among other uses. An editor can give that training a clear target by choosing a change the comparison must catch. Source: LVMH.

The tooling layer: platforms, agents, and workflows

Google calls its measures pass@k and pass^k. The first counts a task as a success if at least one of k attempts succeeds. The second requires all of them to succeed. One good suggestion may be enough for a brainstorming session. A buyer about to approve an order gets little comfort from knowing the tool found the new price yesterday if it misses it today. Source: Google ADK.

A recurring meeting also lends itself to a simpler option. With Gemini in Apps Script, a Google Workspace user can describe how rows in Sheets should become meeting notes in Docs and get help writing the script. The script runs with the user's permissions. Use the script to copy prices and dates exactly, then ask AI to draft the questions the buyer needs to raise. Source: Gemini in Apps Script.

Governance and risk: what needs to be in place before scaling

In Workspace Studio, an administrator can let a pilot group create private drafts while restricting steps that affect other people. A step that tries to send an email can then fail at runtime. Check that the draft survives and the right person can pick it up. If your organization requires EU processing, distinguish storage from processing: blocking steps that require processing outside the assigned region also disables Studio's generative AI features. Source: Workspace Studio. Source: Data Regions.

For GitHub Enterprise Cloud customers with Enterprise Managed Users, Microsoft Agent 365 can show Copilot prompts, responses, and tool calls. If two runs disagree, a reviewer can check whether the agent fetched different material. Limit log access to the people doing that review. Source: GitHub Copilot.

This week's practical Hammer test

Set aside 40 minutes with the colleague who approves purchases. Use two versions of a quote you've already reviewed.

  1. Write the answer key, 10 minutes. Note the old and new delivery dates, one unchanged price and a question the supplier left open. Keep the key to yourself.
  2. Run it three times, 10 minutes. Give the same two files to the same model in three fresh conversations. Use this instruction each time:

Compare the quotes before our meeting. Show the old and new values for each changed term and tell me where to find them. End with questions the documents leave open. Give me meeting notes I can review.

  1. Check the work, 15 minutes. Did all three answers get the date right? Did any describe the unchanged price as new? Could you find the evidence in the quote? Write down the exact term each run missed.
  2. Decide what happens next, 5 minutes. If one run misses the date, have the next version copy the date field directly from the source and try again. If all three hold up, the buyer can try the method before the next meeting. The buyer still approves the notes and sends the reply to the supplier.

Companies and tools to watch

  • Apollo Hospitals: Time to close a reviewed case is a useful measure well beyond healthcare. Source: Apollo Hospitals.
  • Le Groupe Figaro: The publisher plans to introduce ChatGPT Enterprise internally. That's a separate commitment from making its journalism available in ChatGPT. Source: Le Groupe Figaro.
  • Google Drive: Try a proposed rule on easily confused documents before letting the resulting labels determine how files are handled. Source: Google Drive.

If the comparison holds up but someone still copies figures by hand every Monday, that's a good job for Tool Forge: connect the quote to the meeting template and leave the final decision with the buyer.

The Forge newsletter

Get new articles in your inbox

Pick the topics you care about. No noise, at most one email a week.

Get new articles in your inbox

We follow GDPR. Unsubscribe anytime.