We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Tracing and using data lineage for pipeline processing in Astro-WISE.
- Authors
Mwebaze, Johnson; Boxhoorn, Danny; Valentijn, Edwin
- Abstract
Most workflow systems that support data provenance primarily focus on tracing lineage of data. Data provenance by data lineage provides the derivation history of data including information about services and input data that contributed to the creation of a data product. We show that tracing lineage by means of full backward chaining not only enables users to share, discover and reuse the data, but also supports scientific data processing through storage, retrieval and (re)processing of digitized scientific data. In this paper, we present Astro-WISE, a distributed system for processing, analyzing and disseminating wide field imaging astronomical data. We show how Astro-WISE traces lineage of data and how it facilitates data processing, retrieval, storage and archiving. Particularly we show how it solves issues related to the changing data items typical for the scientific environment, such as physical changes in calibrations, our insight in these changes and improved methods for deriving results.
- Subjects
DATA pipelining; INFORMATION storage &; retrieval systems; ASTRONOMY databases; ELECTRONIC data processing; SCIENCE databases; DATA analysis; ASTRONOMY
- Publication
Experimental Astronomy, 2013, Vol 35, Issue 1/2, p131
- ISSN
0922-6435
- Publication type
Article
- DOI
10.1007/s10686-011-9276-8