Image: ChatGPT AI-generated illustration

Source: APQC in alliance with Cognitive World

The Importance of Structured Knowledge

Organizations are structuring and curating knowledge so AI tools can find and present the right answers, grounding responses in validated sources instead of plausible-sounding hallucinations. In a RAG implementation, that source material is often the company's own knowledge base and internal documentation, information that doubles as competitive advantage. For retrieval to work, this information must be curated and structured.

Componentized Content: The Backbone of AI Retrieval

The practice behind that structuring is called componentized content. Instead of one long manual, the material is broken into discrete components (a single topic, a group of topics, or even a fragment of content like a warning or reference table), each carrying its own metadata. This allows content to be authored, tagged, versioned, and reused independently versus staying buried inside a 400-page PDF. Technical communication teams have built this discipline under structured authoring standards like DITA1. AI/ML engineers have a similar structural task to create embeddings. While both disciplines use similar terms like chunk or topic, their approaches and outcomes are fundamentally different.

Where Language Gets in the Way

That is where things get confusing. People often use the same words to describe different things. An AI engineer's idea of a component is not always a knowledge architect's idea of a component, or a technical writer's idea of a topic. Each defines the term in the way that is most useful to their own work, and those definitions can diverge sharply. In the best case, the gap gets caught and clarified in one conversation. In the worst case, it stalls a project for months before anyone traces the failure back to a word nobody had agreed on.

I watched this happen on a recent call with one of our AI engineers, a knowledge architect, and a technical documentation specialist. We were walking through how our platform breaks a technical manual into pieces a retrieval system can use. Partway through, I said something like: “When I think about these components, I mean a chunk of text. An answer to a question.”

Our AI/ML engineer said, in effect: “Sure, these are topic entities, concepts, not answers.” He continued to explain that if I wanted to split the document by paragraph, we could easily change the chunking algorithm in the settings. (Imagine a confused look on my face.)

We were on the same call, showing the same platform, using the same word. We meant two different things.

If you have participated in knowledge management or information architecture work, you have probably watched some version of this happen. It happens when a KM team starts evaluating an AI search or retrieval tool. Everyone nods along when terms like chunk, topic, granularity, context, and semantic are used. Everyone leaves the meeting thinking they agreed on something. They agreed on a word. Nobody discussed the architecture behind it, and nobody finds out until they see an outcome that doesn’t match what the business truly needs.

Technical communicators and knowledge managers learned these words in one discipline. AI and ML engineers learned them in another. Both are using the correct definition for their field. That is what makes the gap easy to miss and expensive to resolve once it surfaces.

Terms with Multiple Meanings

Five terms cause most of the damage:

“We're doing semantic chunking” is a true sentence often said by both, but it describes two different pipelines. In one, semantic chunking is algorithm-driven; in the other, it reflects a content strategy. Neither side is wrong, and neither side will notice the mismatch until the retrieval results come back wrong and everyone starts blaming the model.

Three Questions to Clarify Definitions

Before your next AI vendor meeting or internal architecture review, write your own two-column glossary for those five terms. Then bring three questions into the room:

When you say chunk or topic, do you mean a unit we authored under an information type and authoring standard, or a slice of text your algorithm produced from a size setting?
When an answer depends on a related component (a prerequisite, a warning, or a cross-reference), does the system follow that relationship, or does context only mean whatever text fits in the model's window?
Is granularity a standard we set once in authoring, or a setting you tune against test results, and when it changes, does our typed markup and taxonomy metadata survive into the index, or is meaning rebuilt purely as vector similarity?

None of these take a technical background to ask. All three will tell you, in about ninety seconds, whose architecture is being described.

Why Definitions Matter

The governance gap caused by mismatched definitions is nothing new. KM practitioners have spent years bridging divides between departments with different taxonomies. With AI, that cost lands faster: a pilot can burn through a full budget cycle optimizing for the wrong definition of chunk before anyone checks the results against what the business asked for.

Most AI pilots stall on knowledge readiness long before they stall on model quality. That readiness starts with agreeing on what the words mean before anyone begins to build the application.

Authors

Seth Earley is an expert with 20+ years’ experience in Knowledge Strategy, Data and Information Architecture, Search-based Applications and Information Findability solutions. Seth has worked with a diverse roster of Fortune 1000 companies helping them to achieve higher levels of operating performance by making information more findable, usable and valuable through integrated enterprise architectures supporting analytics, e-commerce and customer experience applications.His work centers around information architecture (IA) and he coined the industry catchphrase “There’s No AI without IA”. Seth Earley is a sought-after speaker, writer, and influencer. His writing has appeared in IT Professional Magazine from the IEEE where, as former editor, he wrote a regular column on data analytics and information access issues and trends. He has also contributed to the Harvard Business Review, CMSWire, Journal of Applied Marketing Analytics, and he co-authored “Practical Knowledge Management” from IBM Press. Seth is author of the award-winning book The AI-Powered Enterprise: Harness the Power of Ontologies to Make Your Business Smarter, Faster and More Profitable. Seth was named to Thinkers360 top 50 global thought leaders and influencers on Artificial Intelligence for 2022 and as a top thought leader for 2023. His current research is in knowledge management and large language models (LLMs)

Heather Eisenbraun

Heather Eisenbraun is Chief AI Knowledge Architect at Earley Information Science, where she designs the ontologies, taxonomies, and knowledge engineering processes that make enterprise content AI-ready.

AI Program Semantics: The Terminology Gap