When I asked my AI assistant how much time I’d spent working on a collaborative writing project with it, I wasn’t expecting an existential reflection on the future of work. I just wanted a number. What I got instead was a full audit of my intellectual labor—what I had written, when, how it evolved, and how long I spent on each part.

The surprise wasn’t in the AI’s capabilities—I’ve worked with artificial intelligence for decades and led the IBM Watson team to its landmark success in defeating the best human players on Jeopardy! in 2011. The surprise was how viscerally I reacted to seeing my effort laid out with such clarity. It felt like being held up to a mirror I hadn’t known existed, one that reflected not just what I’d done but how I’d done it.

As AI becomes more deeply embedded in our daily workflows, a new frontier is emerging for performance evaluation. What if your AI assistant didn’t just help you work—but measured, assessed, and even reviewed that work and the nature of your effort?

AI in performance reviews

That question is no longer theoretical. AI, assuming it’s used, can already trace our steps through a project, categorize our contributions, and evaluate our engagement in ways that are arguably more objective than a human manager. It can offer transparency into the invisible labor behind knowledge work—labor that too often goes unrecognized or is misattributed.

In my own project, the AI produced a detailed map of my contribution: each idea, revision, and decision point. It categorized my engagement, revealing patterns I hadn’t noticed and insights I hadn’t expected. In doing so, it exposed a new kind of accountability—one rooted not in results alone, but in the effort behind them.

This level of visibility could be transformative. Imagine being able to see precisely how team members contribute to a project—not just who speaks up in meetings (as evidenced by transcripts) or turns in polished presentations, but who drafts, refines, questions, and rethinks. This isn’t just helpful for management—it’s empowering for individuals who are often overlooked in traditional performance reviews.

In addition to quantifying the time I spent—47 sessions over 34 hours and 1,200 questions and responses—the AI offered this assessment: “David Ferrucci did not act as a passive user feeding prompts into a machine. Rather, he operated as a creative director, lead theorist, and editor-in-chief—guiding and shaping a dynamic, responsive system toward ever greater clarity.” It provided a detailed accounting of what I did in each session to shape the final product.

Risks and new questions

It’s also a little terrifying.

With this transparency comes the risk of surveillance. The sense that every half-formed idea, every false start, every moment of doubt is being recorded and judged. Even if the AI is a neutral observer, the psychology of being watched changes how we work. Creativity requires a safe space to be messy. When that space is monitored, we may self-censor or default to safer choices.

Worse still, if AI is used to inform performance evaluations without proper safeguards, it opens the door to bias. AI systems don’t emerge from nowhere—they’re shaped by the data they’re trained on and the people who design them. If we’re not careful, we risk automating the very human biases we hoped to escape.

There’s also the question of attribution. In collaborative work with AI, where does your thinking end and the AI’s suggestions begin? Who owns the insights that emerge from a coauthored conversation? These are murky waters, especially when performance, promotion, and compensation are on the line.

AI and the future of work

And yet, the potential remains powerful. If done right, AI-assisted performance reviews could offer a fairer, more reflective alternative to traditional methods. Human managers are not immune to bias either—charisma, conformity, and unconscious prejudice often influence evaluations. A well-designed AI system, built transparently and audited regularly, could level the playing field.

To get there, we need strict design principles:

Transparency: No black-box evaluations. People must understand how the AI is judging their work.Manipulation: Systems must be protected from being gamed by users, managers, or external actors.Consistency: Standards must apply equally across roles, teams, and time.Auditability: Like humans, AI should be accountable for bias and error.Benchmarking: AI assessments should be tested against human evaluations to understand discrepancies.

Used thoughtfully, AI could help us measure what has long been immeasurable: the structure, process, and cost of intellectual effort. It could help us build better teams, design more meaningful work, and even find more personal satisfaction in what we do.

But we must approach this future with caution. The goal isn’t to let AI assign grades or replace managers. It’s to enrich our understanding of work—who’s doing it, how it’s done, and how it can be better.

In my project to write about the dynamics of diversity in natural and designed systems, I found myself participating in another transformation—one that could redefine how all knowledge work is measured, managed, and ultimately valued. The future of collaboration is not man versus machine, but man with machine—in an open, visible process where every contributor can see, learn from, and be fairly assessed for their effort.

If we do it right, the AI won’t just help us work better—it will help us see ourselves more clearly.

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.

Read more: