Data Quality Monitoring Reports - SemWebQuality.org

Data Quality Monitoring Reports

From SemWebQuality.org
(Difference between revisions)
Jump to: navigation, search
(Example 10: Expiry Rule (OWL DL Design))
Line 41: Line 41:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
[[Create Data Requirements#Example 1: PropertyCompletenessRule (Minimal Input)|Click here to learn how to define a data requirement for this report]]
+
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]
  
 
===Example 1.2: Property Completeness Violations (OWL DL Design)===
 
===Example 1.2: Property Completeness Violations (OWL DL Design)===
Line 74: Line 74:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
[[Create Data Requirements#Example 1: PropertyCompletenessRule (Minimal Input)|Click here to learn how to define a data requirement for this report]]
+
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]
  
 
===Example 2: Conditional Property Completeness Violations (OWL DL Design)===
 
===Example 2: Conditional Property Completeness Violations (OWL DL Design)===
Line 116: Line 116:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule|Click here to learn how to define a data requirement for this report]]
+
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]
  
 
===Example 3: Syntax Rule Violations (OWL DL Design)===
 
===Example 3: Syntax Rule Violations (OWL DL Design)===
Line 176: Line 176:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]
  
 
===Example 5: Legal Value Range Rule (OWL DL Design)===
 
===Example 5: Legal Value Range Rule (OWL DL Design)===
Line 207: Line 209:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]
  
 
===Example 6: Legal Value Rule (OWL DL Design)===
 
===Example 6: Legal Value Rule (OWL DL Design)===
Line 242: Line 246:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]
  
 
===Example 7: Unique Value Rule (OWL DL Design)===
 
===Example 7: Unique Value Rule (OWL DL Design)===
Line 268: Line 274:
 
GROUP BY ?dqr ?i
 
GROUP BY ?dqr ?i
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]
 +
  
 
===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL Design)===
 
===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL Design)===
Line 312: Line 321:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]
  
 
===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL Design)===
 
===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL Design)===
Line 359: Line 370:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]
  
 
===Example 10: Expiry Rule (OWL DL Design)===
 
===Example 10: Expiry Rule (OWL DL Design)===
Line 385: Line 398:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]
 +
  
 
===Example 11: Update Rule (OWL DL Design)===
 
===Example 11: Update Rule (OWL DL Design)===
Line 412: Line 428:
 
}
 
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

Revision as of 20:40, 20 October 2011

This site is currently under construction!
Problem reports are an excellent instrument to quickly identify instances with requirement violations.

Contents

Prerequisites

With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:

Data Requirement Violation Reports with SPARQL

Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of dqm:TestedClass and dqm:TestedProperty (See Create Data Requirements#Define Tested Elements for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.


Example 1.1: Property Completeness Violations (OWL Full Design)

Scenario: Find all instances of a certain class that miss a property or a property value.
Input: All property completeness rules (in OWL Full Design)
Output: All instances with missing values
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:PropertyCompletenessRule .
    ?dqr dqm:testedClass ?tclass .
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:requiredValue "true"^^xsd:boolean .
    ?dqr dqm:requiredProperty "true"^^xsd:boolean .
    {    
        ?i a ?tclass .
        NOT EXISTS{
           ?i ?tprop ?value .
        }
    }UNION{
        ?i a ?tclass .
        ?i ?tprop "" .
    }
}

Click here to learn how to define a data requirement for this report

Example 1.2: Property Completeness Violations (OWL DL Design)

Scenario: Find all instances that miss a property or a property value.
Input: All property completeness rules (in OWL DL Design)
Output: All instances with missing values
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:PropertyCompletenessRule .
    ?dqr dqm:testedClass ?tclass .
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:requiredValue "true"^^xsd:boolean .
    ?dqr dqm:requiredProperty "true"^^xsd:boolean .
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    {    
        ?i a ?tclassURI .
        NOT EXISTS{
           ?i ?tpropURI ?value .
        }
    }UNION{
        ?i a ?tclassURI .
        ?i ?tpropURI "" .
    }
}

Click here to learn how to define a data requirement for this report

Example 2: Conditional Property Completeness Violations (OWL DL Design)

Scenario: Find all instances in a subset of a certain class that miss a property or a property value.
Input: All conditional property completeness rules (in OWL DL Design)
Output: Instances with missing values
SELECT ?dqr ?i ?condvalue
WHERE {
    ?dqr a dqm:ConditionalPropertyCompletenessRule .
    ?dqr dqm:testedClass ?tclass . 
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:requiredProperty "true"^^xsd:boolean .
    ?dqr dqm:requiredValue "true"^^xsd:boolean .
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    ?dqr dqm:hasCondition ?cond1 .
    ?cond1 dqm:conditionalProperty ?cprop .
    ?cprop dqm:hasURI ?cpropreal .
    ?cond1 dqm:equals ?condvalue .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
    {
       ?i a ?tclassURI .
       ?i ?cpropURI ?value 
       NOT EXISTS{
          ?i ?tpropURI ?value1 .
      }
    }UNION{ 
       ?i a ?tclassURI .
       ?i ?cpropURI ?value .
       ?i ?tpropURI "" .
    }
    FILTER (str(?value) = str(?condvalue)) .
}

Click here to learn how to define a data requirement for this report

Example 3: Syntax Rule Violations (OWL DL Design)

Scenario: Find all instances of a certain class with property values that violate a syntax rule.
Input: All syntax rules (in OWL DL Design)
Output: All instances with property values that have syntax violations
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:SyntaxRule .
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:testedClass ?tclass . 
    ?dqr dqm:regex ?regex .
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    ?i a ?tclassURI .
    ?i ?tpropURI ?value .
    FILTER (!regex(str(?value), ?regex)) .
}

Click here to learn how to define a data requirement for this report

Example 4: Conditional Syntax Rule Violations (1 Condition, OWL DL Design)

Scenario: Find all syntax violations of property values in a subset of a certain class.
Input: All conditional syntax rules(in OWL DL Design).
Output: All instances of the subset of the class that have syntax violations.
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:ConditionalSyntaxRule .
    ?dqr dqm:testedClass ?tclass . 
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:regex ?regex .
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    ?dqr dqm:hasCondition1 ?cond1 .
    ?cond1 dqm:conditionalProperty ?cprop .
    ?cprop dqm:hasURI ?cpropreal .
    ?cond1 dqm:equals ?condvalue .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
    ?i a ?tclassURI .
    ?i ?cpropURI ?condvalueplain .
    ?i ?tpropURI ?value .
    FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .
}

Click here to learn how to define a data requirement for this report

Example 5: Legal Value Range Rule (OWL DL Design)

Scenario: Find all instances that have property values out of a specified range.
Input: All legal value range rules (in OWL DL Design)
Output: All instances with out of range values
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:LegalValueRangeRule .
    ?dqr dqm:testedClass ?tclass . 
    ?dqr dqm:testedProperty1 ?tprop .
    OPTIONAL{
      ?dqr dqm:upperLimit ?upperLimit .
    }
    OPTIONAL{
      ?dqr dqm:lowerLimit ?lowerLimit .
    }
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    ?i a ?tclassURI .
    ?i ?tpropURI ?value .
    FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || 
    STRDT(str(?value), xsd:float) < ?lowerLimit) .
}

Click here to learn how to define a data requirement for this report

Example 6: Legal Value Rule (OWL DL Design)

Scenario: Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.
Input: All legal value rules (in OWL DL Design).
Output: All instances with values not listed in the trusted reference.
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:LegalValueRule .
    ?dqr dqm:testedClass ?tclass .
    ?dqr dqm:testedProperty1 ?tprop . 
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    ?dqr dqm:referenceClass ?rclass .
    ?dqr dqm:referenceProperty1 ?rprop .
    ?rclass dqm:hasURI ?rclassvalue .
    ?rprop dqm:hasURI ?rpropvalue .
    BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
    BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
    ?i a ?tclassURI .
    ?i ?tpropURI ?value .
    OPTIONAL {
        ?i2 a ?rclassURI .
        ?i2 ?rpropURI ?value1 .
        FILTER (str(?value1) = str(?value)) .
    } .
    FILTER (!bound(?value1)) .
}

Click here to learn how to define a data requirement for this report

Example 7: Unique Value Rule (OWL DL Design)

Scenario: Find all instances with property values that are not unique although they should.
Input: All unique value rules (in OWL DL Design).
Output: All instances with non-unique values of property that must have unique values for each instance.
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:UniqueValueRule .
    ?dqr dqm:testedClass ?tclass .
    ?dqr dqm:testedProperty1 ?tprop . 
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    ?i a ?tclassURI .
    ?i ?tpropURI ?uniqueValue1 .
    ?i2 ?tpropURI ?uniqueValue2 .
    FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}
GROUP BY ?dqr ?i

Click here to learn how to define a data requirement for this report


Example 8: Functional Dependency Value Rule (1 Condition, OWL DL Design)

Scenario: Find all instances with inconsistent property values.
Input: All functional dependency value rules with 1 condition (in OWL DL Design).
Output: All instances with inconsistent property values.
SELECT ?dqr ?i ?dvalue ?value
WHERE {
    ?dqr a dqm:FuncDepValueRule .
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:testedClass ?tclass . 
    ?dqr dqm:equals ?dvalue .
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    ?dqr dqm:hasCondition1 ?cond1 .
    ?cond1 dqm:conditionalProperty ?cprop1 .
    ?cprop1 dqm:hasURI ?cpropreal1 .
    ?cond1 dqm:equals ?condvalue1 .
  MINUS{
   ?dqr dqm:hasCondition2 ?cond2 .
  }
  MINUS{
   ?dqr dqm:hasCondition3 ?cond3 .
  }
  MINUS{
   ?dqr dqm:hasCondition4 ?cond4 .
  }
  MINUS{
   ?dqr dqm:hasCondition5 ?cond5 .
  }
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
    ?i a ?tclassURI .
    ?i ?cpropURI1 ?condvalue1 .
    ?i ?tpropURI ?value .
    FILTER (str(?dvalue) != str(?value)) .
}

Click here to learn how to define a data requirement for this report

Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL Design)

Scenario: Find all instances with inconsistent property values.
Input: All functional dependency value rules with 2 conditions (in OWL DL Design).
Output: All instances with inconsistent property values.
SELECT ?dqr ?i 
WHERE {
    ?dqr a dqm:FuncDepValueRule .
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:testedClass ?tclass . 
    ?dqr dqm:equals ?dvalue .
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    ?dqr dqm:hasCondition1 ?cond1 .
    ?cond1 dqm:conditionalProperty ?cprop1 .
    ?cprop1 dqm:hasURI ?cpropreal1 .
    ?cond1 dqm:equals ?condvalue1 .
    ?dqr dqm:hasCondition2 ?cond2 .
    ?cond2 dqm:conditionalProperty ?cprop2 .
    ?cprop2 dqm:hasURI ?cpropreal2 .
    ?cond2 dqm:equals ?condvalue2 .
  MINUS{
   ?dqr dqm:hasCondition3 ?cond3 .
  }
  MINUS{
   ?dqr dqm:hasCondition4 ?cond4 .
  }
  MINUS{
   ?dqr dqm:hasCondition5 ?cond5 .
  }
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
    BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .
    ?i a ?tclassURI .
    ?i ?cpropURI1 ?condvalue1 .
    ?i ?cpropURI2 ?condvalue2 .
    ?i ?tpropURI ?value .
    FILTER (str(?dvalue) != str(?value)) .
}

Click here to learn how to define a data requirement for this report

Example 10: Expiry Rule (OWL DL Design)

Scenario: Find all outdated instances.
Input: All expiry rules (in OWL DL Design) and validity date of instances.
Output: All instances with that are not valid anymore.
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:ExpiryRule .
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:testedClass ?tclass . 
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    BIND ((now()) AS ?today)
    ?i a ?tclassURI .
    ?i ?tpropURI ?expiry .
    FILTER(?expiry<?today)
}

Click here to learn how to define a data requirement for this report


Example 11: Update Rule (OWL DL Design)

Scenario: Find all outdated instances.
Input: All update rules (in OWL DL Design) and timestamps of instances.
Output: All instances with that are potentially outdated.
SELECT ?dqr ?i
WHERE {
    ?dqr a dqm:UpdateRule .
    ?dqr dqm:testedClass ?tclass .
    ?dqr dqm:testedProperty1 ?tprop .
    ?dqr dqm:expectedUpdateInterval ?duration .
    ?tclass dqm:hasURI ?tclassreal .
    ?tprop dqm:hasURI ?tpropreal .
    BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
    BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
    BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
    ?i a ?tclassURI .
    ?i ?tpropURI ?timestamp .
    FILTER(?reqTimestamp>?timestamp)
}

Click here to learn how to define a data requirement for this report

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox