The Data Tier Is the Hard Part

The first three parts of this series established a progression. Modernization is constrained by production proof, not by code-generation volume. That proof weakens when a platform cannot cover the full language and dependency profile of the estate, and generative AI, valuable for acceleration, cannot by itself provide the repeatable execution required for production cutover.

Part 4 moves to the layer where defects are often hardest to detect and most expensive to correct: the data tier. Code defects are visible in demonstrations and tests. Data defects can remain structurally invisible until production exposes a changed balance, sequence, relationship, or service level, sometimes not becoming apparent until a quarterly report reveals the discrepancy.

A program can transform application code correctly and still fail. The target architecture may be clean, the converted logic may compile and execute, and every row may load. Yet cutover can still miss because the data either fails to preserve the required source business meaning or does not correctly implement an approved change to that meaning. It may also fail to reconcile to expected business outcomes or perform within the required operating window.

Better code conversion does not eliminate the failure boundary; it shifts the primary challenge towards the data semantics, system behavior, and business intent. This is where the architecture behind the AI matters. General-purpose LLMs can accelerate analysis and generate plausible mappings, but they do not, by themselves, provide authoritative knowledge of IMS hierarchy, VSAM key order, Db2 isolation behavior, or the business consequences embedded in those constructs.

mLogica addresses that gap between general-purpose AI capabilities and the specialized knowledge required for legacy modernization through domain-specific SLMs and a governed, deterministic rule layer designed for legacy modernization. The SLMs accelerate discovery and mapping; deterministic execution makes transformations versioned, repeatable, and inspectable; validation converts technical output into evidence. The distinction may appear small at the syntax level. It becomes material in the data tier, where meaning, not syntax, must survive.

Many late-stage modernization failures originate not in code syntax, but in data semantics, reconciliation gaps, performance degradation, and drift in transaction behavior.

Consider a recurring pattern in modernization projects: data passes every technical validation check, yet subtle changes in business meaning go undetected. For example, a legacy field uses null values to indicate “not yet known” or that a particular field has not been reviewed or adjusted. During migration, the target schema, following reasonable relational practice, replaces null with a default value such as zero. Every row loads. Row counts match. Schema validation passes. The migration appears complete.

Then a monthly batch runs. A downstream calculation that interpreted null as "not yet known" now interprets the value as "known to be zero." As a result, some accounts may now settle differently than they did on the source. Nothing crashes, and no log reports an error. The divergence is real, financially consequential, and introduced by a schema decision that looked defensible in isolation.

That is the shape of the risk. The most dangerous data-tier defects are not the ones that fail loudly. They are the ones that pass structural checks and quietly change a business outcome. Structural validation cannot expose them. Behavioral reconciliation can.

Legacy data stores encode operating behavior alongside their contents: field meanings, default values, sort sequences, key structures, precision rules, access paths, locking assumptions, commit boundaries, restart points, and exceptions that may never have been documented outside the running system.

• In IMS and IDMS, hierarchy and navigation are part of the application contract; flattening those models into relational tables can preserve values while changing sequence, cardinality, or update behavior
• In VSAM, alternate indexes, record layouts, and key order can matter as much as the data itself
• In Db2, outcomes can depend on isolation levels, cursor semantics, stored procedures, and optimizer behavior that do not transfer automatically to a cloud target

The same principle applies beyond the mainframe. Teradata, Sybase, Netezza, and Informix each carry vendor-specific data types, functions, distribution methods, temporary-table behavior, and workload assumptions.

Data migration is therefore not an export-and-load workstream. It is the controlled preservation, or intentional redesign, of business meaning through transformation.

A modernized system should not always reproduce the legacy system exactly. The purpose may be to remove accumulated data debt, replace an obsolete locking model, correct a defective precision rule, convert an overnight batch into a stream, or remodel an entity that was constrained by the original platform. In those cases, identical output may preserve the very problem the program was intended to solve.

The discipline is not fidelity for its own sake. It is evidence-based accountability for every difference. Each variance between source and target must be classified as either an approved, documented, intentional change or an unexplained divergence that must be resolved. A program that cannot separate the two is not governing change; it is guessing. Reconciliation should not force the target to imitate the source. It should prevent intended improvement from being confused with silent corruption.

A target schema can look correct while violating rules embedded in the source. A null can become a default, changing the distinction between “unknown” and “zero.” A numeric conversion can introduce rounding differences that affect calculations and reporting. A collation change can alter sort order. A generated key can break relationships with downstream references. Each change may pass schema review and still alter a business result.

Preserving referential integrity becomes more difficult during phased modernization. Historical data may be loaded in bulk while current transactions are replicated through change data capture, causing parent and child records to follow different migration or synchronization paths. Relationships that were implicitly maintained in the source system must therefore be explicitly preserved throughout the migration.

Even when referential integrity is maintained, behavioral consistency is not guaranteed. Deletes, reversals, late-arriving records, duplicates, and out-of-order updates must all be handled consistently across the two systems. The issue is not simply whether the target contains the same data. It is whether the target preserves the source system's business meaning and operating behavior, except where a change has been intentionally designed and explicitly approved.

Change data capture and parallel-run architectures reduce cutover risk by synchronizing source and target and comparing results before the legacy system is retired. They are essential controls, but they are not proof by themselves.

A CDC pipeline will propagate an incorrect mapping as quickly and consistently as a correct one. A parallel run can show that outputs differ without identifying the transformation rule, schema decision, timing condition, or transaction sequence that caused the divergence. Replication lag, event ordering, conflict resolution, and source-of-truth rules can create false mismatches or conceal real ones. Synchronization shows that two systems are moving together. It does not establish that either result is correct.

This is where determinism earns its place. A transformation is deterministic when the same input, under the same versioned rule set, always produces the same output, and any change can be traced to a specific, inspectable rule. That is stronger than automation. It means a failed validation can be traced to the responsible rule, corrected once, regenerated across affected workloads, and retested consistently. Ad hoc manual patches and model-generated output without a governed rule layer undermine that reproducibility, leaving evidence fragmented and difficult to defend.

Row counts are a starting point, not a verdict. Production-grade reconciliation must compare record completeness, control totals, referential relationships, transaction outcomes, financial balances, batch outputs, exceptions, and downstream reports. It must also accommodate approved schema evolution while isolating every variance that has not been approved.

The strongest test is properly scoped output equivalence. Given the same business event, does the modernized system produce the required result, update the same logical entities, preserve the required audit history, and satisfy the same downstream obligations? Where the result is intentionally different, is that difference approved, documented, and traceable to the decision that authorized it? Without business-level reconciliation, a migration can pass every technical check and still fail the enterprise.

A correct answer delivered after the service-level window has closed is not operationally equivalent to the legacy system. Formal correctness and performance are distinct properties, but the business requires both. A result that arrives too late can disrupt settlement, reporting, customer service, or the next dependent batch just as effectively as a wrong result.

Legacy platforms were engineered around predictable throughput, optimized access paths, and controlled batch windows. Modernization changes that performance model. Network latency, distributed joins, storage behavior, indexing, commit frequency, concurrency, and optimizer choices can turn a functionally correct conversion into an operational bottleneck. Production-representative testing must validate throughput, sustained capacity, restart behavior, timing, and sequence before cutover, not after it.

The industry often separates application modernization from database modernization because they are sold and staffed as different workstreams. Production does not honor that separation. Application logic determines how data is created, interpreted, updated, and consumed; data structures constrain how the application can behave; batch schedules, interfaces, reports, and operating procedures bind the two together. Transform one without governing the other, and the program creates an integration seam precisely where cutover risk is highest.

A complete architecture maintains traceability from recovered business logic through schema evolution, data conversion, application transformation, reconciliation, and performance validation. mLogica’s LIBER*M and STAR*M are built around this combined model, keeping mainframe application and dependency transformation coordinated with distributed database and workload modernization. TRAK*M AutoTest provides the validation layer that compares source and target behavior rather than assuming that conversion has proven equivalence. Migration patterns may vary by workload; the requirement for a connected, traceable evidence chain does not.

Not every table warrants the same level of ceremony. Full behavioral reconciliation, production-scale performance proof, and a governed evidence chain are appropriate where a silent error would be expensive, regulated, or irreversible, including financial balances, audit trails, customer-facing calculations, and safety-relevant records. A low-risk reference table does not require the same treatment. The discipline is to scale proof to consequence and defend, workload by workload, why the selected level of evidence is sufficient.

One limitation remains: reconciliation can only validate against a known or recoverable answer. Where intended behavior exists only in an undocumented COBOL paragraph or in the memory of a retired subject-matter expert, the hardest task is recovering the rule itself. Tooling can accelerate that discovery and make the recovered logic inspectable; it cannot substitute for it. Programs that treat knowledge recovery as a formal workstream are less likely to be surprised at cutover.

1. Semantics: How will the program prove that business meaning, not merely field values, survives the migration, and how will it distinguish intentional change from accidental divergence?
2. Transformation: Are schema and data mappings versioned, inspectable, repeatable, and governed through deterministic execution, so a correction can be applied once and validated consistently?
3. Reconciliation: Can the platform compare transaction outcomes, balances, batch outputs, exceptions, and downstream reports rather than relying primarily on row counts?
4. Performance: How will throughput, concurrency, batch windows, restart behavior, and service levels be validated with production-representative volumes?
5. Accountability: When source and target differ, can the team trace the variance to the responsible rule and correct it systematically, or does each discrepancy become a manual investigation?

Cloud providers, database vendors, and systems integrators have made cloud landing zones easier to provision and scale. That progress is necessary, but it does not resolve the hardest part of modernization. Beyond the landing zone, the modernization program still needs a governed capability to transform, reconcile, and validate the legacy data tier against source-system behavior, business rules, and operational baselines. Without this governed validation capability, the workload may successfully move to the cloud while data correctness remains unproven. The delivery risk has changed, not disappeared.

Modernization is strongest when code behavior and data behavior are proven together. AI can accelerate discovery and mapping. Deterministic pipelines can govern transformation. Automated reconciliation can identify and account for differences at scale. Performance validation can establish operational readiness. Used together, and calibrated to the consequence of each workload, these controls replace assumption with evidence. The data tier is where that evidence must hold, because it is where business meaning, transaction integrity, and production performance converge.

Next in this series: Proof Before Cutover, why parallel run and evidence packages are becoming the release gate for mission-critical modernization.

Peter Burney is Senior Vice President of Strategic Programs at mLogica, where he leads enterprise growth initiatives spanning go-to-market strategy, executive partnerships, strategic alliances, and cross-divisional programs. He has been central to mLogica's evolution as an AI-governed, deterministic modernization platform company, working with global systems integrators, cloud platform providers, and enterprises in regulated industries.