Originally published in Russian on Habr (MWS company blog), October 1, 2026.

Cover illustration from the original Habr post.

I’m the product owner of MWS Data Scout, a service that uses large language models (LLMs) to describe database tables.

In this post I’ll explain why simply handing the schema to a model doesn’t solve the documentation problem, which layers of context we had to build, and which prompt block, contrary to what we expected, only makes the output worse.

Why describing tables is hard

MWS Data Scout started with a single experiment. We took the Resource Inventory system and had seven employees describe its data marts by hand.

Result after three months: 310 data marts.

Speed: about two tables per person per day.

Time spent: up to six hours per table.

It takes that long because answering “what’s in here, and why?” is hard: you have to find the data owner, reconstruct the system’s history, figure out how status_2 differs from status_new, and check your guesses with SQL queries.

At MWS scale — one of our clusters alone holds more than 600 databases — you can’t describe the data landscape by hand. It grows faster than people can document it.

When we started, AI agents working on top of enterprise data didn’t exist yet. The goal was practical: help the analyst who spends hours, every time, figuring out which of forty similar tables holds the current data. We only found out later that a ready-made layer of descriptions would come in handy for AI agents too.

First attempt: hand the model the schema

We started with the most obvious approach and gave the LLM the structure of the data: table names, column names, and data types.

The result was disappointing. The model just restated what was already visible in the schema: “The table contains information about users; the user_id column is the user identifier.” There was no new knowledge in texts like that, and no business logic.

The problem is that a column name is just a label a developer gave it at some point. A technical field called status_2 could turn out to be a payment status, a delivery status, or a migration flag. You can’t infer that directly from the schema.

Layers of context

To get past this barrier, we started collecting context from every available source:

1. Data samples. We take 10 random rows per table. That’s not much for analysis, but it’s enough to tell a reference table from a log, and to see that a column holds specific, recurring values rather than “some statuses.”

2. Classification by usage. We split tables into types:

• Log — events and monitoring
• Transactional — business transactions and ETL (including staging tables)
• Directory — reference tables with fixed values
• Bridge — many-to-many relationships
• Summary — aggregates and summary metrics

You have to describe a log and a reference table in completely different ways, so the table class is the first thing we determine.

3. Business domain. If a company doesn’t have its own description of its business domain, we fall back on industry standards. For telecom that’s eTOM (a process map for telecom operators); for finance, BIAN. As a result, the model matches each table against a vetted external map of the business.

4. A custom glossary. If there’s no ready-made industry model, the product builds a business glossary for the domain from the data itself and maps the tables to it.

5. Documentation. We hooked up a parser for Confluence, git repositories, and files (PDF, DOCX, XLSX, MD, TXT, HTML). Pages are split into chunks, stored in Elasticsearch, and retrieved with hybrid search — full-text and vector search at the same time.

What works in a prompt, and what gets in the way

You’d think the more context we give the model, the better the result. The experiments showed the opposite.

How we tested

We used a single model throughout and ran 1,048 generations for 93 unique tables from three different sources. An automatic rater — LLM-as-a-judge combined with CatBoost — scored quality on a scale from 1 to 5. We measured the effect of each prompt block with a paired t-test and a fixed-effects regression.

We tested these prompt blocks:

• task — the instruction: what needs to be done
• examples — samples of the desired output
• output_format — a strict format specification
• documentation — the retrieved documentation chunks

Effect of each block on description quality

Prompt block	Quality gain	p-value
task	+0.60	<0.001
examples	+0.51	<0.001
output_format	−0.08	0.012
documentation	−0.11	0.020

The top two results were expected. The bottom two were not: a strict format specification and added documentation both make quality worse. And documentation extraction was where we had put in the most effort!

Why this happens

Both blocks add noise:

• Documentation often contains outdated information, or describes a neighboring system or a different process. The model starts blindly copying it into the description.
• The format specification pulls too much of the model’s attention away from analyzing the data itself.

Combinations of blocks are even more interesting. The best quality came from context, examples, and documentation together. But adding the task block to that set lowered the overall score. A block that’s useful on its own only gets in the way in an overloaded prompt.

The main takeaway: you can’t assemble a prompt on the “let’s throw in everything we have” principle. We ended up moving to a prompt matrix: each combination of table class and business domain gets its own version of the prompt. A prompt for a financial reference table and a prompt for a customer click log are different things, and they evolve independently.

Results

We tested the system on production databases at MTS (the Russian telecom group MWS belongs to). Results of the automation:

• 8,180 tables from 7 DBMSs were fully described in less than 3 days.
• Generation took about 25 seconds per table (instead of 6 hours of manual work).
• About 80% of the documentation work is now fully automated. All the team has to do is set up the domain and approve the result.
• Analysts accept up to 95% of the descriptions with no edits at all.
• One person’s throughput went up from 30 to 2,000 tables a month.

Speed depends heavily on how much context has been collected. One system with 634 tables took us 33 hours; another with 4,653 tables took just 1.75 hours. That’s pure generation time, not counting human review.

Besides the descriptions themselves, the system handles related tasks:

• Builds ER diagrams, including hidden relationships that it detects by analyzing overlaps in the data itself (when the relationships aren’t declared as foreign keys).
• Generates business terms and synonyms.
• Labels personal data (PII).

Protecting personal data

Labeling personal data is critical, and mistakes are expensive. Since May 2025, data leaks have carried large revenue-based fines under Federal Law No. 152-FZ (Russia’s personal data law): up to 3% of annual revenue (up to 500 million rubles) for a repeat leak and up to 15 million rubles for a first one.

The biggest danger is “forgotten” copies in legacy systems. Our product automatically searches the entire data landscape for personal data, using LLMs and classical machine learning. From the ER diagram we can see exactly where that data spreads across storage.

Important: sensitive fields are masked at the network gateway before an AI agent can access them. The data itself never leaves the company’s secure perimeter.

Technical limitations

1. Data sources. More than eight database connectors are available out of the box: Oracle, PostgreSQL, MS SQL, ClickHouse, MySQL, Greenplum, Hadoop. Lakehouse architecture is supported, and there’s integration with the DataHub and OpenMetadata data catalogs. Results can be exported to Excel and JSON.

2. Cost and infrastructure. Every LLM call costs money. The models are deployed inside the company perimeter, on dedicated servers with A100 and A40 GPUs. That’s a fixed cost and, at the same time, a performance ceiling: to scale, you have to buy more of that expensive hardware.

3. Optimization. To cut costs, we’re moving simple tasks (PII detection, for example) from heavy LLMs to lightweight classifiers — anything from logistic regression to DistilBERT-class models. We train them on AI-generated synthetic data. This brings response time down to milliseconds and gives stable, deterministic results.

MWS Data Scout for AI agents

We built the product to help people find the information they need faster, stop spending time figuring out tables, and stop creating duplicates. But once enterprise AI agents arrived, it turned out they need the semantic layer even more.

Unlike a person, an AI agent can’t just walk over and “ask a colleague for directions.” It can only work with information that’s documented clearly and systematically.

Without clear descriptions, a glossary, and relationships, an AI agent is flying blind: it mixes up tables among hundreds of similar ones, interprets business metrics (say, “revenue”) differently every time, joins tables at random, and can accidentally expose confidential data.

Summary

MWS Data Scout gathers context in layers: data samples, table classification, business domain, a glossary, and documentation. Beyond text descriptions, the system builds ER diagrams (including hidden relationships), generates business terms, and labels personal data. With it, one person’s throughput went from 30 to 2,000 tables a month.

AI deserves a separate mention. Agents can confuse similar tables, interpret metrics arbitrarily, join data at random, and risk exposing confidential information. Our product lets you create a single, accurate source of descriptions of all your databases for other AI agents to use. And the sensitive data itself stays inside the secure perimeter.