Before You Download the Numbers: A Practical Test for Public Data Portals Public data portals can make a difficult question look easy. Two sites may offer a table with the same familiar label, yet one counts applications while the other counts approved cases. One may report calendar years, another fiscal years, and a third a rolling twelve-month period. Downloading the cleanest spreadsheet first can create a precise-looking answer to a question the data never measured. A better approach is to compare portals before collecting a large file. Define the decision, inspect how each source describes its units, test a small sample, and record why one dataset fits better than the alternatives. This guide is for students, community researchers, volunteer analysts, and small organizations that need a defensible shortlist without pretending that every public number is directly comparable.
PUBLIC DATA PORTAL TEST
1
Begin with the decision, not the portal search box Write the decision the data will support in one sentence. “Understand local housing” is too broad. “Compare the annual number of completed rental units across three districts from 2021 through 2025” identifies an event, geography, measure, and period. A strong question makes unsuitable datasets easier to reject. Then define the unit of analysis. Is one row a person, household, application, inspection, property, business, trip, or monthly total? Similar-looking charts can count different units. A portal that reports inspections cannot answer how many distinct buildings were inspected unless it also supplies a reliable building identifier and the method permits deduplication. State the level of precision the decision needs. A neighborhood planning discussion may need a directional trend, while a funding allocation may require a documented count and revision history. Do not collect more detail than the decision can responsibly use. Fine-grained location or personal attributes can increase privacy risk without improving the answer. Create a four-line question card before searching: •
Decision: what someone will do differently after seeing the result.
•
Measure: the event, condition, or quantity being compared.
•
Boundary: geography, population, and time period.
•
Minimum evidence: the fields and documentation required to answer honestly. This card is a stop rule. If a portal cannot meet the minimum evidence, move it to a “possible context” list instead of forcing it into the main comparison.
Compare definitions before comparing values Dataset titles are navigation labels, not complete definitions. Open the metadata, data dictionary, methodology note, or field descriptions. Find the exact inclusion rule, exclusions, update process, and treatment of duplicates. If those materials are missing, record that absence as a limitation rather than filling the gap from the title. Translate each candidate measure into a plain statement: “one row represents…” and “a record enters this dataset when…”. These two sentences reveal many mismatches. For example, a permit dataset may add a record when an application is filed, when a permit is issued, or when work is completed. All three can be called “permits” in a chart, but they describe different stages. Watch for changing definitions inside one time series. A category may be renamed, a threshold revised, or a new reporting system introduced. A sudden rise can reflect coverage rather than a real-world change. Look for break markers, revision notes, archived dictionaries, or fields that indicate the reporting version. Use a compact comparison record for each portal: 1
Copy the displayed dataset title and version date.
2
Write the unit of analysis in your own words.
3
Record the inclusion event and important exclusions.
4
Note whether duplicates are possible and how they are handled.
5
Mark definition changes that affect the requested years. Do not average or merge values until these statements are compatible. If two measures answer different questions, present them separately even when combining them would produce a smoother chart.
PUBLIC DATA PORTAL TEST
2
Match geography, time, and population on the same terms Geographic names can hide different boundaries. A city label might mean the legal municipality, postal area, service region, police district, school zone, or a statistical approximation. Save the boundary name, code, and edition. If a portal provides a boundary file, check whether its year matches the records being summarized. Time fields need similar care. Separate the date an event occurred from the date it was reported, processed, published, or revised. Choose the field that matches the question and keep that choice consistent. State whether a period is calendar year, fiscal year, academic year, quarter, month, or rolling interval. Population filters may be explicit or buried in the collection process. A service dataset describes people who reached that service, not necessarily everyone who needed it. A survey may exclude groups by age, language, housing situation, or contact method. An online complaint system reflects access to the reporting channel as well as the underlying issue. Before accepting a candidate, answer these checks: •
Do the geographic units cover the same physical area?
•
Do the time periods start and end on comparable dates?
•
Does the population definition remain stable across the series?
•
Are suppressed, missing, estimated, and zero values distinguishable?
•
Could an administrative change alter who appears in the data? If a boundary or population cannot be reconciled, narrow the claim. “Records received by this system” is often more accurate than “all events in the community.”
Inspect provenance, revisions, and the actual export Identify who creates the records, who publishes them, and whether those roles differ. A portal may redistribute data produced by another agency. Preserve the producing organization, publication page, dataset identifier, update date, and license or reuse statement. Do not infer permission from the presence of a download button. Check whether the portal replaces earlier files or preserves versions. A value may change after late reports, corrections, deduplication, or reclassification. Record the retrieval date and any visible revision label. If the project will be repeated, keep the original downloaded file and a checksum so a later analyst can tell whether the source changed. Preview the export before committing to it. Confirm file format, character encoding, delimiter, column names, data types, and row granularity. A portal preview may show formatted labels while the export contains codes that require a separate lookup table. Download the dictionary and code list at the same time as the data when they are necessary to interpret the fields. Treat blank, zero, suppressed, not applicable, and unknown as different states. Replacing all of them with zero creates false certainty. If a portal uses special symbols or negative values as missing-data codes, preserve those codes until the documentation explains them. Finally, inspect one ordinary row and one edge case. Trace each field back to its definition. This small test often catches shifted columns, mixed date formats, broken characters, duplicated headers, or totals embedded among detail rows before they spread through an analysis.
PUBLIC DATA PORTAL TEST
3
Use discovery pages to find candidates, not to certify them Search engines, bookmarks, institutional lists, and link collections can shorten the discovery stage. A categorized reference such as 주소온길 사이트모음 may help identify possible websites to inspect, but it does not establish who owns a destination, whether its dataset is current, or whether the data fits your question. Open every candidate independently. Confirm the final domain after redirects, identify the publisher, and locate the dataset's own metadata and reuse terms. Keep the discovery route separate from the evidence record. The page that helped you find a portal should not be cited as the authority for a definition maintained elsewhere. Stop if a destination unexpectedly requests payment, account recovery, broad permissions, or personal information. Return through a known institutional path or contact the responsible publisher. Do not upload sensitive files or create invented records merely to test an export workflow. When several candidates remain, rank them by fit rather than visual polish. A plain download page with clear definitions, stable identifiers, and revision notes may be more useful than an interactive dashboard whose underlying measure is unclear.
Test a small sample and leave a decision card Download the smallest slice that can test your question: one period, one geography, or a limited export if the portal permits it. Check row count, unique identifiers, missing states, date range, and category values. Compare a few published totals only when the portal explains that both views use the same filters and revision. Run the intended calculation on the sample. If the decision needs a rate, verify that numerator and denominator cover the same population, geography, and period. If it needs a trend, confirm that the measure and reporting system remain comparable. Do not build a full pipeline until the sample behaves as the documentation predicts. Close the review with a decision card: •
Selected dataset: title, producer, identifier, and checked version.
•
Fit: the question it can answer under stated conditions.
•
Known limits: missing coverage, definition breaks, suppression, or revision risk.
•
Rejected alternatives: one sentence explaining each mismatch.
•
Recheck trigger: a new release, changed dictionary, boundary revision, or failed sample test. Keep the rejected alternatives. They show that the selected portal was chosen by criteria rather than convenience and prevent the same unsuitable source from being reconsidered without new evidence. If no candidate meets the minimum evidence, report that result and revise the question. An honest “not comparable” conclusion is more useful than a confident table built from mismatched measures.
PUBLIC DATA PORTAL TEST
4
Frequently asked questions Is the newest dataset always the best choice? No. A recent file may use a new definition that breaks comparison with earlier years. Choose the version that fits the question and document any break. Recency matters after scope, measure, and coverage align.
Can dashboard totals be copied instead of downloading data? Sometimes, if the question requires only those displayed totals and the dashboard documents its filters, definitions, and revision date. Record the exact view and check date. Use an export when you need row-level verification, repeatable filtering, or a preserved version.
What if two portals publish different values for the same label? Compare their units, inclusion events, geography, time fields, revision dates, and suppression rules before treating the difference as an error. If the definitions remain incompatible, report both measures separately and name what each one counts. A reliable dataset shortlist begins before the download. By fixing the question, comparing definitions, matching boundaries, checking provenance, and testing a small sample, you can choose a source that supports a bounded claim and leave enough evidence for someone else to repeat the decision.
PUBLIC DATA PORTAL TEST
5