Data Quality Monitoring Reports
Problem reports are an excellent instrument to quickly identify instances with requirement violations.
Prerequisites
With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
- Declare the elements of your ontology that shall be used in the DQM-Vocabulary
- Create Data Requirements
SPARQL-Problem-Reports
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of dqm:TestedClass and dqm:TestedProperty (See Create Data Requirements#Define Tested Elements for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.
Example 1.1: Property Completeness Violations (OWL DL Design)
Scenario: Find all instances that miss a property or a property value.
Input: All property completeness rules
Output: All instances with missing values
SELECT ?dqr ?i WHERE { ?dqr a dqm:PropertyCompletenessRule . ?dqr dqm:testedClass ?tclass . ?dqr dqm:testedProperty1 ?tprop . ?dqr dqm:requiredValue "true"^^xsd:boolean . ?dqr dqm:requiredProperty "true"^^xsd:boolean . ?tclass dqm:hasURI ?tclassreal . ?tprop dqm:hasURI ?tpropreal . BIND (IRI(str(?tpropreal)) AS ?tpropURI) . BIND (IRI(str(?tclassreal)) AS ?tclassURI) . { ?i a ?tclassURI . NOT EXISTS{ ?i ?tpropURI ?value . } }UNION{ ?i a ?tclassURI . ?i ?tpropURI "" . } }
Click here to learn how to define a data requirement for this report
Example ´1.2: Property Completeness Violations (OWL Full Design)
Scenario: Find all instances that miss a property or a property value.
Input: All property completeness rules (in OWL Full Design)
Output: All instances with missing values
SELECT ?dqr ?i WHERE { ?dqr a dqm:PropertyCompletenessRule . ?dqr dqm:testedClass ?tclass . ?dqr dqm:testedProperty1 ?tprop . ?dqr dqm:requiredValue "true"^^xsd:boolean . ?dqr dqm:requiredProperty "true"^^xsd:boolean . { ?i a ?tclass . NOT EXISTS{ ?i ?tprop ?value . } }UNION{ ?i a ?tclass . ?i ?tprop "" . } }
Click here to learn how to define a data requirement for this report
Example 2: Conditional Property Completeness Violations (OWL DL Design)
SELECT ?dqr ?i ?condvalue WHERE { ?dqr a dqm:ConditionalPropertyCompletenessRule . ?dqr dqm:testedClass ?tclass . ?dqr dqm:testedProperty1 ?tprop . ?dqr dqm:requiredProperty "true"^^xsd:boolean . ?dqr dqm:requiredValue "true"^^xsd:boolean . ?tclass dqm:hasURI ?tclassreal . ?tprop dqm:hasURI ?tpropreal . ?dqr dqm:hasCondition ?cond1 . ?cond1 dqm:conditionalProperty ?cprop . ?cprop dqm:hasURI ?cpropreal . ?cond1 dqm:equals ?condvalue . BIND (IRI(str(?tclassreal)) AS ?tclassURI) . BIND (IRI(str(?tpropreal)) AS ?tpropURI) . BIND (IRI(str(?cpropreal)) AS ?cpropURI) . { ?i a ?tclassURI . ?i ?cpropURI ?value NOT EXISTS{ ?i ?tpropURI ?value1 . } }UNION{ ?i a ?tclassURI . ?i ?cpropURI ?value . ?i ?tpropURI "" . } FILTER (str(?value) = str(?condvalue)) . }
Click here to learn how to define a data requirement for this report
Example 3: Syntax Rule Violations (OWL DL Design)
SELECT ?dqr ?i WHERE { ?dqr a dqm:SyntaxRule . ?dqr dqm:testedProperty1 ?tprop . ?dqr dqm:testedClass ?tclass . ?dqr dqm:regex ?regex . ?tclass dqm:hasURI ?tclassreal . ?tprop dqm:hasURI ?tpropreal . BIND (IRI(str(?tpropreal)) AS ?tpropURI) . BIND (IRI(str(?tclassreal)) AS ?tclassURI) . ?i a ?tclassURI . ?i ?tpropURI ?value . FILTER (!regex(str(?value), ?regex)) . }
Click here to learn how to define a data requirement for this report
Example 4: Conditional Syntax Rule Violations
SELECT ?dqr ?i WHERE { ?dqr a dqm:ConditionalSyntaxRule . ?dqr dqm:testedClass ?tclass . ?dqr dqm:testedProperty1 ?tprop . ?dqr dqm:regex ?regex . ?tclass dqm:hasURI ?tclassreal . ?tprop dqm:hasURI ?tpropreal . ?dqr dqm:hasCondition1 ?cond1 . ?cond1 dqm:conditionalProperty ?cprop . ?cprop dqm:hasURI ?cpropreal . ?cond1 dqm:equals ?condvalue . BIND (IRI(str(?tpropreal)) AS ?tpropURI) . BIND (IRI(str(?tclassreal)) AS ?tclassURI) . BIND (IRI(str(?cpropreal)) AS ?cpropURI) . ?i a ?tclassURI . ?i ?cpropURI ?condvalueplain . ?i ?tpropURI ?value . FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) . }