Non-parametric Statistical Matching with primary and secondary farm data
What is it about?
This work considers the challenges faced by researchers in the Big Data and the ‘Internet of Things’ era. Indeed, the processes of data collection, storage and use are being transformed, while the relationship between data collected first hand (primary data) and data collected by someone else (secondary data) is becoming more fluid. In this scenario, data integration is emerging as a reliable strategy to overcome data shortage and other challenges (data coverage, quality, time dis-alignment and representativeness). Among the others, Micro Statistical Matching techniques (MiSM) are very promising methods. They have been used in the social sciences, politics and economics, but there are very few applications that use agricultural and farm data. The work presents an example of MiSM data integration between primary and secondary farm data on agricultural holdings in Italy. The novelty of the work lies in the fact that integration is carried out with non-parametric MiSM, which is compared to predictive mean matching and Bayesian linear regression, while the matching validity is assessed with a new strategy. The lessons learned and the use in a research field characterised by critical data shortage are discussed.
Why is it important?
A new, recently updated method is applied for the integration of agricultural data on farms and farm households: the non-parametric Statistical Matching (MiSM). This has never been applied to agricultural data, nor it has seen the application to different primary and secondary data. Moreover, a new validation strategy for the assessment of the synthetic completed (matched) data set generated by MiSM is presented and discussed.