Data curation

In the modern era of big data, the curation of data has become more prominent, particularly for software processing high volume and complex data systems. The term is also used within the humanities, where increasing cultural and scholarly data from digital humanities projects requires the expertise and analytical practices of data curation. In broad terms, curation means a range of activities and processes done to create, manage, maintain, and validate a component. Specifically, data curation is the attempt to determine what information is worth saving and for how long.

History and practice

The user, rather than the database itself, typically initiates data curation and maintains metadata. According to the University of Illinois' Graduate School of Library and Information Science, "Data curation is the active and on-going management of data through its lifecycle of interest and usefulness to scholarship, science, and education; curation activities enable data discovery and retrieval, maintain quality, add value, and provide for re-use over time." The data curation workflow is distinct from data quality management, data protection, lifecycle management, and data movement.

Census data has been available in tabulated punch card form since the early 20th century and has been electronic since the 1960s. The Inter-university Consortium for Political and Social Research (ICPSR) website marks 1962 as the date of their first Survey Data Archive.

Deep background on data libraries appeared in a 1982 issue of the Illinois journal, Library Trends. For historical background on the data archive movement, see "Social Scientific Information Needs for Numeric Data: The Evolution of the International Data Archive Infrastructure." The exact curation process undertaken within any organisation depends on the volume of data, how much noise the data contains, and what the expected future use of the data means to its dissemination.

The crises in space data led to the 1999 creation of the Open Archival Information System (OAIS) model, stewarded by the Consultative Committee for Space Data Systems (CCSDS), which was formed in 1982.

The term data curation is sometimes used in the context of biological databases, where specific biological information is firstly obtained from a range of research articles and then stored within a specific category of database. For instance, information about anti-depressant drugs can be obtained from various sources and, after checking whether they are available as a database or not, they are saved under a drug's database's anti-depressive category. Enterprises are also utilizing data curation within their operational and strategic processes to ensure data quality and accuracy.