When a company commissions new software, everyone focuses on the build. Nobody thinks about what it takes to move the existing data across — until migration day, when it turns out half the records are formatted differently and 20% are duplicates. That surprise adds weeks and budget to projects that were otherwise on track.
Data migration is the part of a software project that gets the least attention in the planning phase and causes the most problems at delivery. Here's what it involves, what it costs, and the questions to ask before you start.
What is data migration?
Data migration is moving your existing records — customer data, order history, product catalogs, financial data, employee records — from the old system to the new one. It sounds like a technical formality. It rarely is.
The complexity comes from three places:
- Data quality — Years of manual entry means duplicates, inconsistent formats, missing fields and values that don't match the new system's rules
- Data volume — Migrating 5,000 records is very different from migrating 5 million
- Data mapping — The old system called it "Klantgroep," the new one calls it "Segment" — and they have different possible values. Every mismatch is a decision
What a migration actually involves
A properly run migration follows these steps:
- Data audit — Before writing a single script, look at what you have. How many records? What percentage are incomplete? Are there duplicates? This step always takes longer than expected.
- Data mapping — Match every field in the old system to a field in the new one. Where there's no match, decide: drop the field, transform it, or flag it for manual review.
- Data cleaning — Fix the quality issues found in the audit. Often the most time-consuming part — and frequently can't be fully automated.
- Migration script — Write and test the script that moves the data.
- Test migration — Run the migration against a copy of production data in a staging environment. Verify that record counts match, spot-check critical records, run reports and confirm the numbers are right.
- Go-live migration — The final run, typically during a maintenance window (overnight or weekend).
- Validation — After go-live, verify the data in the new system is correct before closing down the old one.
What it costs: real numbers
Migration cost is driven primarily by data quality, volume, and how different the old and new data models are:
- Simple migration (clean data, matching fields, under 100,000 records): €2,000 – €6,000
- Medium migration (some cleaning needed, 100K–1M records, moderate mapping complexity): €6,000 – €20,000
- Complex migration (poor data quality, multiple source systems, legacy formats, millions of records): €20,000 – €60,000+
What most project budgets miss: data cleaning. When the audit reveals that 30% of records have missing required fields — fields the new system won't accept — someone has to either fix the data manually or write logic to handle the gaps. This can easily double the estimated migration cost, and it only surfaces after the audit is done. Any quote you receive before that audit is a rough estimate, not a firm price.
Big-bang or phased migration?
There are two structural approaches, and the choice affects your project timeline and risk profile:
Big-bang: Everything moves at once. Old system goes offline, new system goes live. Fast and clean, but high-risk — if something goes wrong with the migrated data, you're dealing with it in production. Best for smaller datasets and systems that can tolerate a few hours of downtime.
Phased: Data moves in stages. Both systems may run in parallel for a period, with data synchronized between them. More complex and more expensive, but significantly lower risk. The right approach for business-critical systems where downtime is not acceptable.
For most Dutch SMBs replacing a core operational system, I recommend the phased approach — the cost of getting it wrong in production is higher than the cost of running parallel systems for a few weeks.
Four risks that catch projects off guard
1. Data that looks clean isn't. A field formatted consistently in the UI may be stored inconsistently in the database. Phone numbers as "0612345678" in one record, "+31612345678" in the next, "06-12 34 56 78" in a third — all mean the same thing, none will match without normalization. This is invisible until you run the first test migration.
2. The new system rejects records you expected to migrate. You discover on test day that the new system requires a postal code in a field that 15% of your old records left empty. Now you need a decision: skip those records, allow blanks, or back-fill from another source. Every such decision takes time, and there are usually many of them.
3. The old system keeps changing during migration. If the business keeps making changes in the old system while migration is in progress, those changes need to be captured and re-migrated — the "delta." The longer a migration takes, the more iterations of this delta sync you'll need. It's one reason to plan for a clean cut-over window rather than an indefinite parallel period.
4. Going back is harder than it looks. Once go-live migration runs and users start entering data in the new system, rolling back to the old one becomes harder by the hour. A failed migration with no rollback plan is a crisis. Define the rollback procedure before you start.
Five questions to settle before migration starts
- What's the actual state of our current data? Request a full export from the old system before the project starts, and look at it. Surprises discovered during a discovery phase are much cheaper to handle than surprises discovered at week eight of development.
- Who is responsible for data cleaning decisions? Developers can write cleaning scripts, but decisions about what to do with missing or ambiguous data require business knowledge. Assign someone on your side — someone who knows what "Klant Type: overig" was supposed to mean.
- Do we need full history, or just current records? Migrating five years of order history is expensive. Migrating only active customers and open orders is not. Be deliberate about what you actually need in the new system versus what can stay in an archive.
- How long can the system be offline? This determines whether you need big-bang or phased, and whether an overnight window is enough or you need a full weekend. Coordinate with operations — a Saturday night migration that runs long is not a good surprise for Monday morning.
- Is the old system staying accessible after go-live? Keep it in read-only mode for at least three months. You will need to look something up. The question isn't whether — it's when.
How migration fits into a software project
Migration planning belongs in the discovery phase, not the final sprint. That's when you audit the existing data, surface risks and produce a realistic cost estimate. A project quote that doesn't address migration either assumes clean data (rarely true) or hasn't considered it (a gap you'll fund later).
If you're replacing a system that was built on spreadsheets, budget for more cleaning than you expect — Excel sheets accumulate years of inconsistencies that databases don't tolerate. If you're replacing a legacy application, add a line item to investigate whether the old system can even export in a usable format. Some older systems can only export to formats that require significant transformation, which adds time before the migration script can even be written.
Done well, migration is invisible — users open the new system and their data is there, correct and complete. Done poorly, it's the thing everyone remembers about the project. It's worth treating it as a first-class deliverable, not a footnote.
If you're planning a system replacement and want to know whether your migration will be a two-day job or a six-week project, get in touch. I'll look at what you have and give you an honest estimate — before you commit to anything.