Articles

Typing it in or importing it: two ways to get data into a system

A form is for one record from the person who knows it; an import is for many, once. What each one fails at, and the count both need afterwards.

By Anes Hamdaoui6 min read
  • data
  • forms
  • migration
All articles

A form is for one record, at the moment it is created, typed by the person who knows it. An import is for many records, once, from a source that already exists. Pick the wrong one and the work lands on the wrong person: an import turns a clerk's five seconds into an afternoon of cleanup, and a form turns three hundred supplier rows into three hundred chances for a typo. Both end the same way — a rule that rejects what cannot be true, and a count that proves what arrived.

What is a form for?

One record, at the moment it happens, by whoever holds the facts. A delivery refused at the door, an expense with the receipt in hand, a repair ticket written while the machine is still open. The form exists because the information is in one head for about ten minutes and then decays into a story.

That gives the form two jobs: capture now, and validate against what the system already knows. The customer number is checked against the customer list; the quantity cannot exceed what the delivery note says is coming. A form that accepts anything is a text box with extra steps.

Its cost is per record, and that is the point: entry is slow on purpose, because that is the only place where a wrong figure can still be argued with. Used for two hundred records, the same form becomes a factory for unverified data.

What is an import for?

Many records, once, from a file that already exists and already means something. Opening balances at go-live, a price list from a supplier, a customer export from the system being replaced, a year of invoices from an accountant. Nobody in the business knows each row; the file is the authority, and the operator's job is to check it, not to retype it.

An import cannot ask a person at row 214 whether the unit is a carton or a piece. It applies rules: this column must be one of these values, this reference must exist, this date must be real, this row must not duplicate a key already in the table. What it cannot decide becomes a rejection or a warning, and the difference between those two is the whole design.

Question Form Import
How many records One Hundreds, once
Who knows the facts The person filling it The file, and whoever made it
What fails silently A plausible wrong figure A whole column mapped to the wrong place

What validation must exist on both paths?

The same three questions in a different order. Is this value possible — a negative quantity, a date in 1900, a four-digit phone number? Is it consistent with the system — does the account exist, is the item in the catalogue? Is it a duplicate — one supplier registered twice under two spellings.

On a form, the third question is answered by a search before the record is created, which is why the deciding field comes first. On an import, it is answered by a key: one column, or a pair, that must be unique, checked against the table before anything is written.

The rule that matters on both paths is the same, and it is about honesty: never replace a value you do not understand with a default. A blank weight is a blank weight; a zero weight becomes a parcel that ships for free and nobody notices for a month.

What has to be reconciled after either one?

Two numbers and one total. How many records were created, how many were rejected, and whether the sum of the money column matches the source file. Without those three, "the import ran" is a claim about a process rather than a statement about data.

For a form, the same discipline applies at a smaller scale: at the end of a shift, the number of forms should match the number of movements the person expected. A form that was started and abandoned is a record nobody has, and it should appear in a short list rather than vanish.

Keep the source file. An import that cannot be explained six months later — which file, which person, which evening — is indistinguishable from data that was invented, and the first audit will ask.

A decision procedure

  1. Count the records. One or a handful is a form. A file of rows is an import. There is no middle case worth building.
  2. For a form, ask who holds the facts and at what moment. If the answer is "the office, next week", it is not a form.
  3. For an import, read twenty rows at random. If half the columns are unintelligible, no tool will fix the file.
  4. Write the validation rules before either path: possible, consistent, unique.
  5. Decide what happens to a rejected row — a queue with an owner, not a log line nobody reads.
  6. Import into a copy, count what arrived, compare the total against the source.
  7. Only then run it on the real table, and keep the file and the count with the date and the name.

Where this goes wrong

  • A spreadsheet nobody validated is imported as though it were a decision. Six months of "temporary" rows, three spellings of one supplier, prices typed with a comma in one column and a point in another. The import copies the mess faithfully and gives it a database.
  • Three hundred supplier rows are typed by hand. Two days at a keyboard produce a result worse than the file, because typing adds errors without adding checks.
  • The import avoids a conversation. Nobody agrees what a customer is, so the export becomes the definition, and every duplicate the business already had is now permanent.
  • The form is used as a bulk entry screen. A clerk with forty deliveries and one form is doing data entry at the speed of a web page.
  • Rejections are counted as tool failures. They are the tool working. The number to watch is rejections nobody resolves.
  • The total is never compared. The file and the table agree only because nobody subtracted the rejected rows, which is how a gap hides.

What this does not cover

This is about getting records in, not designing the table they land in, and it assumes the file can be read at all: a scanned paper archive needs different work before either path applies. Nor does it cover both at once — opening balances arrive by import and then change daily by form, which is fine while the import runs once and the form owns the record afterwards. And nothing here decides whether the system should hold the data at all.

We build both paths against the same rules — a form for the record someone is holding, an import for the file that already exists, a count and a total after either.

Services in this article

Start a project

Have something to build?

One short brief is enough. You get a written scope and a fixed quote before anything starts.