FOM Chapter 12 - Data Processing

Chapter 12

DATA PROCESSING One of the major activities of the 2022 Census of Agriculture and Fisheries (CAF) is processing data from the accomplished census questionnaires. This phase of the census comes immediately after the conduct of data collection. This chapter explains the flow of data processing from the Regional Statistical Services Offices (RSSOs)/Provincial Statistical Offices (PSOs) to the Central Office (CO).

1.

12.1 OVERVIEW OF THE 2022 CAF DATA PROCESSING

Processing census data provides an opportunity in determining the suitability and challenges of the decentralized phases of processing of data in PSOs and RSSOs based on Computer-Assisted Personal Interviewing (CAPI) and Paper-and-Pencil Personal Interviewing (PAPI), the timing of the processing phases and defined responsibilities of project staff. This also tests the efficiency of the data processing, monitoring, and tabulation systems/programs to be used and tables to be generated. A carefully planned data evaluation and tabulation procedures of the 2022 CAF data could reveal possible deficiency of the questionnaire with respect to coverage of items needed.

A four-day task force training and provincial or second level training will be conducted to cover instructions for the data processing of 2022 CAF for the smooth manual and machine processing at the PSO. Table 12.1 shows the timeline for the conduct of data processing trainings, manual/machine processing, data evaluation, further processing as well as its required output.

Table 12.1. Timetable of Activities for the 2022 CAF Data Processing Concerned Activity Date Output Personnel Training of Trainers 12 to 15 Sep. 2023 AFCD, ITDS AFCD, ITDS, NCS Task Force Training 25 to 28 Sep. 2023 Second Level Training Data Processing Evaluation of Marginal and Consistency Tables Evaluation of Key Indicators Certification Pass

Philippine Statistics Authority

16 to 19 Oct. 2023 06 to 30 Nov. 2023 04 Dec. 2023 to 15 Mar. 2024

NCS, ITDS, RSSO, PSO PSO

Cleaned data files

RSSO/PSO

Cleaned/evaluated Tables

RSSO/PSO 11 Dec. 2023 to 15 Mar. 2024

RSSO/PSO

Evaluated Key Indicators Evaluated provinces for certification pass

2022 Census of Agriculture and Fisheries

144

Field Operations Manual | Chapter 12 – Data

Processing

Activity Further Processing

Date 18 Mar. to 20 Apr. CO 2024

Concerned Personnel

Output Preliminary Tables

12.2 STAGES OF THE 2022 CAF DATA PROCESSING 2.

Running tabulation programs for key indicators using the 2022 CAF data will also provide an assessment on whether or not the planned tabulations can be produced easily and/or timely. Time is a factor which should be examined in tabulating the 2022 CAF data considering actual census timetable on data releases. Preliminary and final census results must be made available in a timely manner. A timetable of various phases of processing census data, including requirements of manpower and equipment, can be formulated, tested, and assessed during the census. It is also important to consider alternative processing methods and all their implications, including speed, efficiency, and cost, by preparing statistical tables through different methods.

Figure 12.1 2022 CAF Data Processing

2.

Organizational Set-up of Data Processing at the PSO The PSO-CSS is responsible for the smooth operation of the provincial processing.

He/She will assign the PSO Focal Person, to supervise the 2022 CAF data processing at the PSO. The PSO Focal Person will be responsible in the day-to-day operations of the provincial processing. He/She will be assisted by a hired Data Processing Supervisor, whose main function is to monitor regularly and supervise the data processing activities.

2022 Census of Agriculture and Fisheries

Philippine Statistics Authority

145 The PSO Clerk should record the number of Forms, Form Type, Date Received based on the actual count.

Field Operations Manual | Chapter 12 – Data Processing

The hired data processors will carry out the manual processing activities which includes checking geographic identification and matching the household members in CAF Forms 2, 3, 4, 5, and 6 against CAF Form 1.

After manual processing, the documents shall proceed to the machine processing: data encoding, key verification, geographic identification validation, completeness checking, and consistency checking. The Supervisor will run the completeness check, geo-id validation, and data consistency checking activities.

The PSO which will be responsible for the manual processing of PAPI questionnaires and machine processing of CAPI, and PAPI will have the following activities: A. Manual Processing (PAPI) 1. Verification of geographic identification (ID) and completeness of forms, 2. Checking for consistency between and among data items and completeness of entries in each questionnaire, 3. Checking for legibility and coding of all entries in the questionnaires using Codebook, and 4. Sample verification of manually processed census forms.

B. Machine Processing (PAPI, CAPI) 1.

2.

3.

4.

Data entry of all entries in the PAPI, Sample key verification of encoded questionnaires, ID validation and completeness of entries checking, and Simple consistency checking, verification, and evaluation.

C. Further Processing/Data Evaluation RSSO 1. Certification pass for: a. ID validation and completeness checking of entries b. Simple Consistency Checking 2. Run Complex Consistency editing, 3. Run marginal totals, preliminary tables, and 4. Evaluate preliminary tables vs previous census data and administrative data if available.

D. Further Processing at CO

The CO will have the following data processing activities for each region/province: 1. Certification pass a. Complex consistency editing by RSSOs, b. Marginal and consistency tables

2. Generation of Tables for Evaluation of CAF key indicators Philippine Statistics Authority

2022 Census of Agriculture and Fisheries

146

Field Operations Manual | Chapter 12 – Data

Processing

a. Data validation of agricultural areas using administrative data; and area planted to specific crops and surface area of fish cages, fishponds, fish pens using remote sensing data 3. Generation of Master File for tabulation including computation of weights which should be attached to the merged/reformatted files, 4. Tabulation and Evaluation, and 5. Preparation and Creation of Micro Data File (MDF).

12.3 2022 CAF DATA PROCESSING PROCEDURES The detailed 2022 CAF data processing procedures/steps and its required census personnel are showed in the table below: Table 12.2. 2022 CAF Data Processing Procedures Procedure

Step CAPI 1 Interview and Data Capture a. Range checks b. Skipping instructions/filters 2 Run validation check and generate validation errors (SMALL EDIT) a. Data items b. Range checks c. Skipping instructions d. Data consistency within form 3 Upload to CO Server; back-up in EN's device (SD Card) 4 Download to TS Device for review a. With error, upload to server 5 Log-in to DPS 6 Run validation check on selected EAs and generate validation errors (BIG EDIT) a. Range checks b. Skipping instructions c. Data consistency within form and within forms 7 Compute and modify % errors per form and total % errors per households and encode to DPS

2022 Census of Agriculture and Fisheries

Role EN

EN

EN TS ACAS ACAS

AS

Philippine Statistics Authority

147

Field Operations Manual | Chapter 12 – Data Processing

Step Procedure 8 Return forms with 50% critical errors or more to EN including error lists Return forms with less than 50% critical error to TS including error lists Edit forms without critical errors 9 Download forms returned by ACAS (> 50% error) Download forms returned by ACAS (<=50% error) 10 Verify/edit questionnaires and conduct verification based on error list 11 Upload verified/edited questionnaires to CO Server 12 Run marginal tables 13 Evaluate marginal tables 14 Return household forms to TS/EN for verification/ reinterview 15 If passed step 13, notify RSSO through MIS and/or email for certification pass 16 Run certification pass by barangay 17 Run key/preliminary tables by barangay 18 Evaluate using the following:- 2012 CAF data - other administrative data - AI4CAF Prediction Models 19 Generate/Evaluate key/preliminary tables through workshop with RSSO/PSO PAPI 1 Manually edit/process forms Encode/Key verify through DPS Data Entry/Key Verification 2 Module Run validation check and generate validation errors (SMALL 3 EDIT) a. Range checks b. Skipping instructions c. Data consistency within form 4 Assign processing of forms with errors to DP or field verification to TS/EN

Philippine Statistics Authority

Role AS AS CAS/ACAS TS EN EN/TS EN/TS AS/DP Supervisor/ISA I AS/PSO Focal Person AS PSO Focal Person RSSO Focal Person RSSO Focal Person RSSO Focal Person

CO

Manual Processor Machine Processor DP Supervisor

DP Supervisor/ CAS/ACAS

2022 Census of Agriculture and Fisheries

148

Processing

Step 5

Field Operations Manual | Chapter 12 – Data

Procedure Edit/Process forms with errors

Role Machine Processor

12.4 NARRATIVE REPORT ON PROVINCIAL DATA PROCESSING Following the suggested template, the Data Processing Supervisor, Information Systems Analyst I, Assistant Statistician and PSO Focal Person shall prepare and submit a narrative report on data processing to RSSO Focal Person, copy furnished the PSO-CSS, within 30 days after the last day of data processing in the province. While the consolidated narrative report is to be submitted by RSSO Focal Person to the National Censuses Service (NCS) (Attention: CPS 2022) through email address HYPERLINK "mailto:cps2022caf@psa.gov.ph"cps2022caf@psa.gov.ph, copy furnished RD and SOCD-Chief, within 40 days after the last day of data processing.

Outline of Narrative Report: 1. CAF Forms 1, 2, 3, 4, and 5 processing: a. Problems/issues encountered, and actions taken/decisions processing, b. Quality control and general impression on the quality of data, c. Summary of start and end of processing by barangay, and d. Summary of errors found in sample verification.

made

in

2. Problems encountered and corresponding actions taken on Data Processing (e.g., verification, and compilation of accomplished barangay/EA/block maps).

3. Feedback on the use of Data Capture System for CAPI, DPS Data Entry/Processing, and MIS. 4. Feedback, problems, and corresponding actions taken related to: a. Data Processing Systems/Programs b. Data Processing Equipment (hardware/utilities/electricity/network/internet connection) c. Processor roles d. Processing flow 5. Assessment of the entire census operations (i.e., generally successful or encountered many problems but completed, and others).

6. Best practices, innovation, strategies employed during data processing.

7. Recommendations to improve future data processing activities.

2022 Census of Agriculture and Fisheries

Philippine Statistics Authority