http://semwebquality.org/mediawiki/index.php?title=Special:Contributions/Admin&feed=atom&deletedOnly=&limit=50&target=Admin&topOnly=&year=&month=SemWebQuality.org - User contributions [en]2024-03-28T11:51:03ZFrom SemWebQuality.orgMediaWiki 1.17.0http://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-11-26T16:28:50Z<p>Admin: /* Functional Dependency Violations (2 Reference Properties, OWL DL Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Quality Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find duplicate instances.<br />
|-<br />
|'''Input:'''||Duplicate instance rules with 3 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have identical values for the 3 tested properties.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value11 .<br />
?i2 ?tpropURI1 ?value12 . <br />
?i ?tpropURI2 ?value21 . <br />
?i2 ?tpropURI2 ?value22 . <br />
?i ?tpropURI3 ?value31 . <br />
?i2 ?tpropURI3 ?value32 .<br />
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))<br />
}<br />
GROUP BY ?dqr<br />
</syntaxhighlight><br />
<br />
<br />
===Example 13: Functional Dependency Violations (2 Reference Properties, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find functional dependency violations between two property values.<br />
|-<br />
|'''Input:'''||Functional dependency reference rules with 2 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have violating dependencies between the two tested property values.<br />
|}<br />
<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepReferenceRule .<br />
?dqr dqm:assessment "true"^^xsd:boolean .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:referenceClass ?trefclass . <br />
?trefclass dqm:hasURI ?trefclassreal .<br />
BIND (IRI(str(?trefclassreal)) AS ?trefclassURI) .<br />
?dqr dqm:referenceProperty1 ?trefprop1 .<br />
?trefprop1 dqm:hasURI ?trefpropreal1 .<br />
BIND (IRI(str(?trefpropreal1)) AS ?trefpropURI1) .<br />
?dqr dqm:referenceProperty2 ?trefprop2 .<br />
?trefprop2 dqm:hasURI ?trefpropreal2 .<br />
BIND (IRI(str(?trefpropreal2)) AS ?trefpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value1 .<br />
?i ?tpropURI2 ?value2 .<br />
BIND(STRDT(str(?value1),xsd:string) AS ?value11 ) <br />
BIND(STRDT(str(?value2),xsd:string) AS ?value12 ) <br />
NOT EXISTS{ <br />
?i2 a ?trefclassURI .<br />
?i2 ?trefpropURI1 ?value11 .<br />
?i2 ?trefpropURI2 ?value12 .<br />
}<br />
} <br />
</syntaxhighlight></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-11-26T16:28:32Z<p>Admin: /* Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Quality Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find duplicate instances.<br />
|-<br />
|'''Input:'''||Duplicate instance rules with 3 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have identical values for the 3 tested properties.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value11 .<br />
?i2 ?tpropURI1 ?value12 . <br />
?i ?tpropURI2 ?value21 . <br />
?i2 ?tpropURI2 ?value22 . <br />
?i ?tpropURI3 ?value31 . <br />
?i2 ?tpropURI3 ?value32 .<br />
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))<br />
}<br />
GROUP BY ?dqr<br />
</syntaxhighlight><br />
<br />
<br />
===Functional Dependency Violations (2 Reference Properties, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find functional dependency violations between two property values.<br />
|-<br />
|'''Input:'''||Functional dependency reference rules with 2 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have violating dependencies between the two tested property values.<br />
|}<br />
<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepReferenceRule .<br />
?dqr dqm:assessment "true"^^xsd:boolean .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:referenceClass ?trefclass . <br />
?trefclass dqm:hasURI ?trefclassreal .<br />
BIND (IRI(str(?trefclassreal)) AS ?trefclassURI) .<br />
?dqr dqm:referenceProperty1 ?trefprop1 .<br />
?trefprop1 dqm:hasURI ?trefpropreal1 .<br />
BIND (IRI(str(?trefpropreal1)) AS ?trefpropURI1) .<br />
?dqr dqm:referenceProperty2 ?trefprop2 .<br />
?trefprop2 dqm:hasURI ?trefpropreal2 .<br />
BIND (IRI(str(?trefpropreal2)) AS ?trefpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value1 .<br />
?i ?tpropURI2 ?value2 .<br />
BIND(STRDT(str(?value1),xsd:string) AS ?value11 ) <br />
BIND(STRDT(str(?value2),xsd:string) AS ?value12 ) <br />
NOT EXISTS{ <br />
?i2 a ?trefclassURI .<br />
?i2 ?trefpropURI1 ?value11 .<br />
?i2 ?trefpropURI2 ?value12 .<br />
}<br />
} <br />
</syntaxhighlight></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-11-26T16:27:53Z<p>Admin: /* Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Quality Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find duplicate instances.<br />
|-<br />
|'''Input:'''||Duplicate instance rules with 3 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have identical values for the 3 tested properties.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value11 .<br />
?i2 ?tpropURI1 ?value12 . <br />
?i ?tpropURI2 ?value21 . <br />
?i2 ?tpropURI2 ?value22 . <br />
?i ?tpropURI3 ?value31 . <br />
?i2 ?tpropURI3 ?value32 .<br />
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))<br />
}<br />
GROUP BY ?dqr<br />
</sytnaxhighlight><br />
<br />
<br />
===Functional Dependency Violations (2 Reference Properties, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find functional dependency violations between two property values.<br />
|-<br />
|'''Input:'''||Functional dependency reference rules with 2 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have violating dependencies between the two tested property values.<br />
|}<br />
<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepReferenceRule .<br />
?dqr dqm:assessment "true"^^xsd:boolean .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:referenceClass ?trefclass . <br />
?trefclass dqm:hasURI ?trefclassreal .<br />
BIND (IRI(str(?trefclassreal)) AS ?trefclassURI) .<br />
?dqr dqm:referenceProperty1 ?trefprop1 .<br />
?trefprop1 dqm:hasURI ?trefpropreal1 .<br />
BIND (IRI(str(?trefpropreal1)) AS ?trefpropURI1) .<br />
?dqr dqm:referenceProperty2 ?trefprop2 .<br />
?trefprop2 dqm:hasURI ?trefpropreal2 .<br />
BIND (IRI(str(?trefpropreal2)) AS ?trefpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value1 .<br />
?i ?tpropURI2 ?value2 .<br />
BIND(STRDT(str(?value1),xsd:string) AS ?value11 ) <br />
BIND(STRDT(str(?value2),xsd:string) AS ?value12 ) <br />
NOT EXISTS{ <br />
?i2 a ?trefclassURI .<br />
?i2 ?trefpropURI1 ?value11 .<br />
?i2 ?trefpropURI2 ?value12 .<br />
}<br />
} <br />
</syntaxhighlight></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-11-26T16:27:08Z<p>Admin: /* Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Quality Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find duplicate instances.<br />
|-<br />
|'''Input:'''||Duplicate instance rules with 3 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have identical values for the 3 tested properties.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value11 .<br />
?i2 ?tpropURI1 ?value12 . <br />
?i ?tpropURI2 ?value21 . <br />
?i2 ?tpropURI2 ?value22 . <br />
?i ?tpropURI3 ?value31 . <br />
?i2 ?tpropURI3 ?value32 .<br />
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))<br />
}<br />
GROUP BY ?dqr<br />
</sytnaxhighlight><br />
<br />
<br />
==Functional Dependency Violations (2 Reference Properties, OWL DL Design)==<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find functional dependency violations between two property values.<br />
|-<br />
|'''Input:'''||Functional dependency reference rules with 2 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have violating dependencies between the two tested property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepReferenceRule .<br />
?dqr dqm:assessment "true"^^xsd:boolean .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:referenceClass ?trefclass . <br />
?trefclass dqm:hasURI ?trefclassreal .<br />
BIND (IRI(str(?trefclassreal)) AS ?trefclassURI) .<br />
?dqr dqm:referenceProperty1 ?trefprop1 .<br />
?trefprop1 dqm:hasURI ?trefpropreal1 .<br />
BIND (IRI(str(?trefpropreal1)) AS ?trefpropURI1) .<br />
?dqr dqm:referenceProperty2 ?trefprop2 .<br />
?trefprop2 dqm:hasURI ?trefpropreal2 .<br />
BIND (IRI(str(?trefpropreal2)) AS ?trefpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value1 .<br />
?i ?tpropURI2 ?value2 .<br />
BIND(STRDT(str(?value1),xsd:string) AS ?value11 ) <br />
BIND(STRDT(str(?value2),xsd:string) AS ?value12 ) <br />
NOT EXISTS{ <br />
?i2 a ?trefclassURI .<br />
?i2 ?trefpropURI1 ?value11 .<br />
?i2 ?trefpropURI2 ?value12 .<br />
}<br />
} <br />
</syntaxhighlight></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Assessment_ReportsData Quality Assessment Reports2011-11-26T14:11:42Z<p>Admin: /* Uniqueness in Scope */</p>
<hr />
<div>Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.<br />
<br />
=Prerequisites=<br />
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
=Data Quality Score Calculations=<br />
<br />
==Completeness==<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?completeness)<br />
WHERE {<br />
{<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?s a ?tclassURI .<br />
NOT EXISTS{<br />
?s ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?s a ?tclassURI .<br />
?s ?tpropURI "" .<br />
}<br />
}UNION{<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?s2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
==Uniqueness in Depth==<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Uniqueness Scores per unique value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)<br />
(((?total - ?violations)/?total) AS ?uniquenessDepth)<br />
WHERE {<br />
{<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}UNION{<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i3 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
<br />
==Uniqueness in Scope==<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.<br />
|-<br />
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)<br />
|-<br />
|'''Output:'''||Uniqueness in scope scores for classes<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?tclassURI <br />
((COUNT(?i)/2) AS ?violations)<br />
(COUNT(?i3) AS ?total)<br />
(((?total - ?violations)/?total) AS ?uniquenessScope)<br />
WHERE {<br />
{<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value11 .<br />
?i2 ?tpropURI1 ?value12 . <br />
?i ?tpropURI2 ?value21 . <br />
?i2 ?tpropURI2 ?value22 . <br />
?i ?tpropURI3 ?value31 . <br />
?i2 ?tpropURI3 ?value32 .<br />
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))<br />
}UNION{<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i3 a ?tclassURI .<br />
}<br />
}<br />
GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight><br />
<br />
==Syntactic Accuracy==<br />
<br />
<br />
===Based on Syntax Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}UNION{<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
===Based on Legal Value Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI <br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}UNION{<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
===Based on Legal Value Range Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}UNION{<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
==Semantic Accuracy==<br />
<br />
===Based on Functional Dependent Value Rule (1 Condition)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.<br />
|-<br />
|'''Input:'''||All functional dependent value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?semanticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}UNION{<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i2 a ?tclassURI .<br />
?i2 ?cpropURI1 ?condvalue1 .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight><br />
<br />
==Timeliness==<br />
===Based on Expiry Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Timeliness]] per expiry rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?timeliness)<br />
WHERE {<br />
{<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}UNION{<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight><br />
<br />
===Based on Update Rules===<br />
<span style="color:red">'''Please Note:'''</span> This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances<br />
|-<br />
|'''Output:'''||Score for [[Timeliness]] per update rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?timeliness)<br />
WHERE {<br />
{<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}UNION{<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Assessment_ReportsData Quality Assessment Reports2011-11-26T14:09:56Z<p>Admin: /* Uniqueness in Depth */</p>
<hr />
<div>Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.<br />
<br />
=Prerequisites=<br />
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
=Data Quality Score Calculations=<br />
<br />
==Completeness==<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?completeness)<br />
WHERE {<br />
{<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?s a ?tclassURI .<br />
NOT EXISTS{<br />
?s ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?s a ?tclassURI .<br />
?s ?tpropURI "" .<br />
}<br />
}UNION{<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?s2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
==Uniqueness in Depth==<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Uniqueness Scores per unique value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)<br />
(((?total - ?violations)/?total) AS ?uniquenessDepth)<br />
WHERE {<br />
{<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}UNION{<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i3 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
<br />
==Uniqueness in Scope==<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.<br />
|-<br />
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)<br />
|-<br />
|'''Output:'''||Uniqueness in scope scores for classes<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?tclassURI <br />
((COUNT(?i)/2) AS ?violations)<br />
(COUNT(?i3) AS ?total)<br />
(((?total - ?violations)/?total) AS ?uniquenessScope)<br />
WHERE {<br />
{<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value11 .<br />
?i2 ?tpropURI1 ?value12 . <br />
?i ?tpropURI2 ?value21 . <br />
?i2 ?tpropURI2 ?value22 . <br />
?i ?tpropURI3 ?value31 . <br />
?i2 ?tpropURI3 ?value32 .<br />
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))<br />
}UNION{<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i3 a ?tclassURI .<br />
}<br />
}<br />
GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight><br />
<br />
==Syntactic Accuracy==<br />
<br />
<br />
===Based on Syntax Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}UNION{<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
===Based on Legal Value Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI <br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}UNION{<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
===Based on Legal Value Range Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}UNION{<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
==Semantic Accuracy==<br />
<br />
===Based on Functional Dependent Value Rule (1 Condition)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.<br />
|-<br />
|'''Input:'''||All functional dependent value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?semanticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}UNION{<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i2 a ?tclassURI .<br />
?i2 ?cpropURI1 ?condvalue1 .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight><br />
<br />
==Timeliness==<br />
===Based on Expiry Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Timeliness]] per expiry rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?timeliness)<br />
WHERE {<br />
{<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}UNION{<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight><br />
<br />
===Based on Update Rules===<br />
<span style="color:red">'''Please Note:'''</span> This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances<br />
|-<br />
|'''Output:'''||Score for [[Timeliness]] per update rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?timeliness)<br />
WHERE {<br />
{<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}UNION{<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-11-26T13:59:46Z<p>Admin: /* Example 11: Outdated / Not Updated Values (OWL DL Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Quality Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find duplicate instances.<br />
|-<br />
|'''Input:'''||Duplicate instance rules with 3 tested properties.<br />
|-<br />
|'''Output:'''||All instances that have identical values for the 3 tested properties.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#><br />
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:DuplicateInstanceRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?tclass dqm:hasURI ?tclassreal . <br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:testedProperty1 ?tprop1 .<br />
?tprop1 dqm:hasURI ?tpropreal1 .<br />
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .<br />
?dqr dqm:testedProperty2 ?tprop2 .<br />
?tprop2 dqm:hasURI ?tpropreal2 .<br />
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .<br />
?dqr dqm:testedProperty3 ?tprop3 .<br />
?tprop3 dqm:hasURI ?tpropreal3 .<br />
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI1 ?value11 .<br />
?i2 ?tpropURI1 ?value12 . <br />
?i ?tpropURI2 ?value21 . <br />
?i2 ?tpropURI2 ?value22 . <br />
?i ?tpropURI3 ?value31 . <br />
?i2 ?tpropURI3 ?value32 .<br />
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))<br />
}<br />
GROUP BY ?dqr<br />
</sytnaxhighlight></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-11-26T12:40:39Z<p>Admin: /* Example 11: Outdated / Not Updated Values (OWL DL Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Quality Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-11-04T21:13:58Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
<br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]] <br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
* [http://groups.google.com/group/dqm-vocabulary DQM-Vocabulary Mailing List] <br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Data Quality Management]]<br />
* [[Quality Perception in Information Systems]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Ontology Quality }}<br />
http://ontologydesignpatterns.org/<br /><br />
http://www.ontoclean.org/<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title=Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-11-04T21:13:14Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
<br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]] <br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
* [http://groups.google.com/group/dqm-vocabulary DQM-Vocabulary Mailing List] <br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Data Quality Management]]<br />
* [[Quality Perception in Information Systems]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title=Ontology Quality }}<br />
http://ontologydesignpatterns.org/<br />
http://www.ontoclean.org/<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-10-24T15:53:29Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
<br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]] <br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
* [http://groups.google.com/group/dqm-vocabulary DQM-Vocabulary Mailing List] <br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Data Quality Management]]<br />
* [[Quality Perception in Information Systems]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td colspan="3"><br />
{{BigHeading<br />
|Title=Links to Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-10-24T15:53:17Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
<script type="text/javascript"><br />
<br />
var _gaq = _gaq || [];<br />
_gaq.push(['_setAccount', 'UA-18733154-1']);<br />
_gaq.push(['_trackPageview']);<br />
<br />
(function() {<br />
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;<br />
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';<br />
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);<br />
})();<br />
<br />
</script><br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]] <br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
* [http://groups.google.com/group/dqm-vocabulary DQM-Vocabulary Mailing List] <br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Data Quality Management]]<br />
* [[Quality Perception in Information Systems]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td colspan="3"><br />
{{BigHeading<br />
|Title=Links to Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-22T20:36:54Z<p>Admin: </p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Quality Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-21T13:03:25Z<p>Admin: </p>
<hr />
<div><br />
== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support [[Data Quality]] management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent [[Data Requirements]], i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and [[Data Quality]] scores based on [[Data Requirements]]<br />
* Automated consistency checks between [[Data Requirements]]<br />
* Exchange of [[Data Quality]] information and [[Data Requirements]] on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Vocabulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Expressing Your Own Data Requirements ==<br />
* [[Create Data Requirements]]<br />
* [[Requirement Metadata]]<br />
<br />
== Consuming DQM-Vocabulary Data ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== Data Quality Knowledge Management==<br />
* [[Generate Problem Annotations]]<br />
* [[Generate Score Annotations]]<br />
* [[Data Requirements Consistency Checks]]<br />
<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
<br />
== See also ==<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Quality Perception in Information Systems]]<br />
<br />
== References ==<br />
'''Fürber, Christian and Hepp, Martin''': [http://www.fuerber.com/publications/Fuerber-Hepp-Towards%20a%20vocabulary%20for%20dqm.pdf Towards a Vocabulary for Data Quality Management in Semantic Web Architectures], Proceedings of the 1st International Workshop on Linked Web Data Management (LWDM2011), Uppsala, Sweden, March 25, 2011.</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-10-20T21:13:59Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]] <br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
* [http://groups.google.com/group/dqm-vocabulary DQM-Vocabulary Mailing List] <br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Data Quality Management]]<br />
* [[Quality Perception in Information Systems]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td colspan="3"><br />
{{BigHeading<br />
|Title=Links to Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T21:13:30Z<p>Admin: /* Example 1.1: Missing Properties / Values (OWL Full Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T21:13:01Z<p>Admin: /* Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T21:12:47Z<p>Admin: /* Example 1.2: Missing Properties / Values (OWL DL Design) */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing properties and values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T21:12:02Z<p>Admin: /* Prerequisites */</p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T21:11:38Z<p>Admin: </p>
<hr />
<div>Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-20T21:10:19Z<p>Admin: /* Consuming the DQM-Vocabulary */</p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br />
<br />
== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support [[Data Quality]] management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent [[Data Requirements]], i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and [[Data Quality]] scores based on [[Data Requirements]]<br />
* Automated consistency checks between [[Data Requirements]]<br />
* Exchange of [[Data Quality]] information and [[Data Requirements]] on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Vocabulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Expressing Your Own Data Requirements ==<br />
* [[Create Data Requirements]]<br />
* [[Requirement Metadata]]<br />
<br />
== Consuming DQM-Vocabulary Data ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== Data Quality Knowledge Management==<br />
* [[Generate Problem Annotations]]<br />
* [[Generate Score Annotations]]<br />
* [[Data Requirements Consistency Checks]]<br />
<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
<br />
== See also ==<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Quality Perception in Information Systems]]<br />
<br />
== References ==<br />
'''Fürber, Christian and Hepp, Martin''': [http://www.fuerber.com/publications/Fuerber-Hepp-Towards%20a%20vocabulary%20for%20dqm.pdf Towards a Vocabulary for Data Quality Management in Semantic Web Architectures], Proceedings of the 1st International Workshop on Linked Web Data Management (LWDM2011), Uppsala, Sweden, March 25, 2011.</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-20T21:09:46Z<p>Admin: /* Data Quality Knowledge Management */</p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br />
<br />
== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support [[Data Quality]] management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent [[Data Requirements]], i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and [[Data Quality]] scores based on [[Data Requirements]]<br />
* Automated consistency checks between [[Data Requirements]]<br />
* Exchange of [[Data Quality]] information and [[Data Requirements]] on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Vocabulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Expressing Your Own Data Requirements ==<br />
* [[Create Data Requirements]]<br />
* [[Requirement Metadata]]<br />
<br />
== Consuming the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== Data Quality Knowledge Management==<br />
* [[Generate Problem Annotations]]<br />
* [[Generate Score Annotations]]<br />
* [[Data Requirements Consistency Checks]]<br />
<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
<br />
== See also ==<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Quality Perception in Information Systems]]<br />
<br />
== References ==<br />
'''Fürber, Christian and Hepp, Martin''': [http://www.fuerber.com/publications/Fuerber-Hepp-Towards%20a%20vocabulary%20for%20dqm.pdf Towards a Vocabulary for Data Quality Management in Semantic Web Architectures], Proceedings of the 1st International Workshop on Linked Web Data Management (LWDM2011), Uppsala, Sweden, March 25, 2011.</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-20T21:09:30Z<p>Admin: /* Consuming the DQM-Vocabulary */</p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br />
<br />
== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support [[Data Quality]] management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent [[Data Requirements]], i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and [[Data Quality]] scores based on [[Data Requirements]]<br />
* Automated consistency checks between [[Data Requirements]]<br />
* Exchange of [[Data Quality]] information and [[Data Requirements]] on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Vocabulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Expressing Your Own Data Requirements ==<br />
* [[Create Data Requirements]]<br />
* [[Requirement Metadata]]<br />
<br />
== Consuming the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== Data Quality Knowledge Management==<br />
* [[Generate Problem Annotations]]<br />
* [[Generate Score Annotations]]<br />
* [[Data Requirements Consistency Checks]]<br />
<br />
* [[Data Quality Monitoring Reports|Data Quality Monitoring Reports (<span style="color:red">'''DEPRECATED'''</span>)]]<br />
<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
<br />
== See also ==<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Quality Perception in Information Systems]]<br />
<br />
== References ==<br />
'''Fürber, Christian and Hepp, Martin''': [http://www.fuerber.com/publications/Fuerber-Hepp-Towards%20a%20vocabulary%20for%20dqm.pdf Towards a Vocabulary for Data Quality Management in Semantic Web Architectures], Proceedings of the 1st International Workshop on Linked Web Data Management (LWDM2011), Uppsala, Sweden, March 25, 2011.</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Generate_Data_Quality_Score_ReportsGenerate Data Quality Score Reports2011-10-20T21:09:02Z<p>Admin: moved Generate Data Quality Score Reports to Data Quality Assessment Reports</p>
<hr />
<div>#REDIRECT [[Data Quality Assessment Reports]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Assessment_ReportsData Quality Assessment Reports2011-10-20T21:09:02Z<p>Admin: moved Generate Data Quality Score Reports to Data Quality Assessment Reports</p>
<hr />
<div>Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.<br />
<br />
=Prerequisites=<br />
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
=Data Quality Score Calculations=<br />
<br />
==Completeness==<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?completeness)<br />
WHERE {<br />
{<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?s a ?tclassURI .<br />
NOT EXISTS{<br />
?s ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?s a ?tclassURI .<br />
?s ?tpropURI "" .<br />
}<br />
}UNION{<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?s2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
==Uniqueness in Depth==<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Uniqueness Scores per unique value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)<br />
(((?total - ?violations)/?total) AS ?uniquenessDepth)<br />
WHERE {<br />
{<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}UNION{<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i3 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
==Syntactic Accuracy==<br />
<br />
<br />
===Based on Syntax Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}UNION{<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
===Based on Legal Value Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI <br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}UNION{<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
===Based on Legal Value Range Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?syntacticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}UNION{<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI ?tpropURI<br />
</syntaxhighlight><br />
<br />
==Semantic Accuracy==<br />
<br />
===Based on Functional Dependent Value Rule (1 Condition)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.<br />
|-<br />
|'''Input:'''||All functional dependent value rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?semanticAccuracy)<br />
WHERE {<br />
{<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}UNION{<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i2 a ?tclassURI .<br />
?i2 ?cpropURI1 ?condvalue1 .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight><br />
<br />
==Timeliness==<br />
===Based on Expiry Rules===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Score for [[Timeliness]] per expiry rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?timeliness)<br />
WHERE {<br />
{<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}UNION{<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight><br />
<br />
===Based on Update Rules===<br />
<span style="color:red">'''Please Note:'''</span> This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.<br />
{|class="wikitable"<br />
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances<br />
|-<br />
|'''Output:'''||Score for [[Timeliness]] per update rule<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?tclassURI<br />
(COUNT(?i) AS ?violations) <br />
(COUNT(?i2) AS ?total)<br />
(((?total - ?violations)/?total) AS ?timeliness)<br />
WHERE {<br />
{<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}UNION{<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i2 a ?tclassURI .<br />
} <br />
}GROUP BY ?dqr ?tclassURI<br />
</syntaxhighlight></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Create_Data_RequirementsCreate Data Requirements2011-10-20T21:07:53Z<p>Admin: </p>
<hr />
<div><br />
__TOC__<br />
<br />
== What are Data Requirements? ==<br />
<br />
Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.<br />
<br />
However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.<br />
<br />
== Types of Data Requirements ==<br />
<br />
Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself. <br />
<br />
[[File:Terminology.png|500px|Table to illustrate used terminology]]<br />
<br />
In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are <br />
# data requirements related to the values of a single property (column)<br />
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)<br />
# data requirements related to the instances of a whole class (table)<br />
# data requirements related to the ontology elements (schema)<br />
<br />
With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.<br />
<br />
==Define Tested Elements==<br />
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:<br />
<br />
=== Design Option 1: Classes and Properties as Instances (OWL Full) ===<br />
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].<br />
<pre><br />
foo:MyClass a dqm:TestedClass<br />
</pre><br />
<pre><br />
foo:MyProperty a dqm:TestedProperty<br />
</pre><br />
<span style="color:red;">'''Attention:'''</span> This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.<br />
<br />
=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===<br />
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].<br />
<pre><br />
foo:Class_1 a dqm:TestedClass<br />
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI<br />
</pre><br />
<pre><br />
foo:Property_1 a dqm:TestedProperty<br />
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI<br />
</pre><br />
<br />
== Sample Dataset used in Examples ==<br />
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:<br />
<br />
{|class="wikitable"<br />
!Original Class!!Mapped Instance<br />
|-<br />
|foo:Location||foo:Class_Location<br />
|-<br />
|}<br />
<br />
...with the following datatype properties:<br />
{| class="wikitable"<br />
! Original Property !! Mapped Instance<br />
|-<br />
|foo:LOCID ||foo:Prop_Location_ID<br />
|-<br />
|foo:STREET||foo:Prop_Location_Street<br />
|-<br />
|foo:STREETNO||foo:Prop_Location_Streetno<br />
|-<br />
|foo:ZIP||foo:Prop_Location_ZIP<br />
|-<br />
|foo:CITY||foo:Prop_Location_City<br />
|-<br />
|foo:COUNTRY||foo:Prop_Location_Country<br />
|-<br />
|foo:STATE||foo:Prop_Location_State<br />
|-<br />
|foo:validThrough||foo:Prop_validThrough<br />
|}<br />
<br />
In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.<br />
<br />
==Syntax Of Examples==<br />
<br />
The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].<br />
<br />
== Examples of Data Requirements ==<br />
<br />
===Example 1: Property Completeness Rule===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]<br />
|-<br />
|'''Dimension:'''||[[dqm:PropertyCompleteness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:<br />
<br />
<br />
'''Definition in OWL Full'''<br />
<pre><br />
foo:PropertyCompletenessRule_1<br />
a dqm:PropertyCompletenessRule ;<br />
dqm:testedClass http://www.example.org/MyClass ;<br />
dqm:testedProperty1 http://www.example.org/MyProperty ;<br />
dqm:requiredProperty "true"^^xsd:boolean ;<br />
dqm:requiredValue "true"^^xsd:boolean .<br />
</pre><br />
[[Data Quality Monitoring Reports#Example 1.1: Missing Properties / Values (OWL Full Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".<br />
<br />
If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:<br />
<br />
<br />
'''Definition in OWL-DL'''<br />
<pre><br />
foo:PropertyCompletenessRule_1<br />
a dqm:PropertyCompletenessRule ;<br />
dqm:testedClass foo:Class_1 ;<br />
dqm:testedProperty1 foo:Property_1 ;<br />
dqm:requiredProperty "true"^^xsd:boolean ;<br />
dqm:requiredValue "true"^^xsd:boolean .<br />
</pre><br />
[[Data Quality Monitoring Reports#Example 1.2: Missing Properties / Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".<br />
<br />
===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]<br />
|-<br />
|'''Dimension:'''||[[dqm:PropertyCompleteness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
To define this data requirement, you must perform the following two steps:<br />
<br />
'''1. Define the Condition'''<br />
<br />
<pre><br />
foo:Condition_USA<br />
a dqm:Condition ;<br />
rdfs:label "Condition USA"^^xsd:string ;<br />
dqm:conditionalProperty foo:Prop_Location_Country ;<br />
dqm:equals "USA"^^xsd:string .<br />
</pre><br />
<br />
'''2. Define the Conditional Property Completeness Rule'''<br />
<br />
<pre><br />
foo:ConditionalPropertyCompletenessRule_State<br />
a dqm:ConditionalPropertyCompletenessRule ;<br />
rdfs:label "Conditional property completeness rule State"^^xsd:string ;<br />
dqm:hasCondition1 foo:Condition_USA ;<br />
dqm:requiredProperty "true"^^xsd:boolean ;<br />
dqm:requiredValue "true"^^xsd:boolean ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_State .<br />
</pre><br />
[[Data Quality Monitoring Reports#Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 3: Syntax Rule (OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]<br />
|}<br />
<br />
You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:SyntaxRule_ZIP<br />
a dqm:SyntaxRule ;<br />
rdfs:label "Syntax rule ZIP"^^xsd:string ;<br />
dqm:regex "^[0-9]{5}$"^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Property_ZIP .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 3: Syntax Violations (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
In order to specify a conditional syntax rule, you must perform the following steps:<br />
<br />
'''1.Define Condition'''<br />
<pre><br />
foo:Condition_USA<br />
a dqm:Condition ;<br />
rdfs:label "Condition USA"^^xsd:string ;<br />
dqm:conditionalProperty foo:Prop_Location_Country ;<br />
dqm:equals "USA"^^xsd:string .<br />
</pre><br />
<br />
'''2.Define Conditional Syntax Rule'''<br />
<pre><br />
foo:ConditionalSyntaxRule_State<br />
a dqm:ConditionalSyntaxRule ;<br />
rdfs:label "Conditional syntax rule State"^^xsd:string ;<br />
dqm:hasCondition1 foo:Condition_USA ;<br />
dqm:regex "^[A-Z]{2}$"^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_State .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 5: Legal Value Range Rule (OWL DL)===<br />
{|class="wikitable"<br />
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.<br />
|-<br />
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]<br />
|}<br />
<br />
You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:<br />
<br />
<pre><br />
foo:LegalValueRangeRule_Price<br />
a dqm:LegalValueRangeRule ;<br />
rdfs:label "Legal value range rule Price"^^xsd:string ;<br />
dqm:lowerLimit "0.00"^^xsd:float ;<br />
dqm:testedClass foo:Class_Product ;<br />
dqm:testedProperty1 foo:Prop_Product_Price .<br />
</pre><br />
<br />
The class [[dqm:LegalValueRangeRule]] has the special properties<br />
* [[dqm:lowerLimit]]<br />
* [[dqm:upperLimit]]<br />
<br />
to specify the lowest and/or highest allowed value.<br />
<br />
[[Data Quality Monitoring Reports#Example 5: Out of Range Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 6: Legal Value Rule (OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify a reference property that holds the allowed values.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]<br />
|}<br />
<br />
In order to specify legal values for a specific property, you need to perform the following steps:<br />
<br />
# Create a reference data set or use an existing data set that holds the legal values <br />
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]<br />
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:LegalValueRule_Country<br />
a dqm:LegalValueRule ;<br />
rdfs:label "Legal value rule Country"^^xsd:string ;<br />
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;<br />
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_Country .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 6: Illegal Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 7: Unique Value Rule (OWL DL)===<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that values of a property must be unique.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:PropertyUniqueness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]<br />
|}<br />
<br />
You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:<br />
<br />
<pre><br />
foo:UniqueValueRule_LOCID<br />
a dqm:UniqueValueRule ;<br />
rdfs:label "Unique value rule LOCID"^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_ID .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 7: Uniqueness Violations (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.<br />
|-<br />
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SemanticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
In order to specify the dependency between two property values, you must perform the following steps:<br />
<br />
# Define a condition under which a specific value is always required. <br />
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:FuncDepValueRule_1<br />
a dqm:FuncDepValueRule ;<br />
rdfs:label "Func dep value rule 1"^^xsd:string ;<br />
dqm:equals "USA"^^xsd:string ;<br />
dqm:hasCondition1 foo:Condition_New_York ;<br />
dqm:reqDescription "If the city value is \"New York\" then the country must be <br />
\"USA\"."^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_Country .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.<br />
|-<br />
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SemanticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
In order to specify the dependency between three property values, you must perform the following steps:<br />
<br />
# Define both conditions under which a specific value is always required. <br />
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:FuncDepValueRule_2<br />
a dqm:FuncDepValueRule ;<br />
rdfs:label "Func dep value rule 2"^^xsd:string ;<br />
dqm:equals "NY"^^xsd:string ;<br />
dqm:hasCondition1 foo:Condition_USA ;<br />
dqm:hasCondition2 foo:Condition_New_York ;<br />
dqm:reqDescription "If the city value is \"New York\" and the country value is <br />
\"USA\" then the state must be \"NY\"."^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_State .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 10: Expiry Rule (OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that instances of a specific class expire.<br />
|-<br />
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]<br />
|-<br />
|'''Dimension:'''||[[dqm:Timeliness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:ClassRequirement]]<br />
|}<br />
<br />
You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:ExpiryRule_1<br />
a dqm:ExpiryRule ;<br />
rdfs:label "Expiry rule 1"^^xsd:string ;<br />
dqm:testedClass foo:Class_Product ;<br />
dqm:testedProperty1 foo:Prop_Product_validThrough .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 10: Outdated / Expired Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 11: Update Rule (OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]<br />
|-<br />
|'''Dimension:'''||[[dqm:Timeliness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:ClassRequirement]]<br />
|}<br />
<br />
You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:UpdateRule_Location<br />
a dqm:UpdateRule ;<br />
rdfs:label "Update rule Location"^^xsd:string ;<br />
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_timestamp .<br />
</pre><br />
<br />
<span style="color:red">'''NOTE:'''</span> The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.<br />
<br />
[[Data Quality Monitoring Reports#Example 11: Outdated / Not Updated Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Create_Data_RequirementsCreate Data Requirements2011-10-20T21:06:25Z<p>Admin: </p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br />
<br />
__TOC__<br />
<br />
== What are Data Requirements? ==<br />
<br />
Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.<br />
<br />
However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.<br />
<br />
== Types of Data Requirements ==<br />
<br />
Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself. <br />
<br />
[[File:Terminology.png|500px|Table to illustrate used terminology]]<br />
<br />
In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are <br />
# data requirements related to the values of a single property (column)<br />
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)<br />
# data requirements related to the instances of a whole class (table)<br />
# data requirements related to the ontology elements (schema)<br />
<br />
With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.<br />
<br />
==Define Tested Elements==<br />
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:<br />
<br />
=== Design Option 1: Classes and Properties as Instances (OWL Full) ===<br />
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].<br />
<pre><br />
foo:MyClass a dqm:TestedClass<br />
</pre><br />
<pre><br />
foo:MyProperty a dqm:TestedProperty<br />
</pre><br />
<span style="color:red;">'''Attention:'''</span> This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.<br />
<br />
=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===<br />
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].<br />
<pre><br />
foo:Class_1 a dqm:TestedClass<br />
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI<br />
</pre><br />
<pre><br />
foo:Property_1 a dqm:TestedProperty<br />
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI<br />
</pre><br />
<br />
== Sample Dataset used in Examples ==<br />
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:<br />
<br />
{|class="wikitable"<br />
!Original Class!!Mapped Instance<br />
|-<br />
|foo:Location||foo:Class_Location<br />
|-<br />
|}<br />
<br />
...with the following datatype properties:<br />
{| class="wikitable"<br />
! Original Property !! Mapped Instance<br />
|-<br />
|foo:LOCID ||foo:Prop_Location_ID<br />
|-<br />
|foo:STREET||foo:Prop_Location_Street<br />
|-<br />
|foo:STREETNO||foo:Prop_Location_Streetno<br />
|-<br />
|foo:ZIP||foo:Prop_Location_ZIP<br />
|-<br />
|foo:CITY||foo:Prop_Location_City<br />
|-<br />
|foo:COUNTRY||foo:Prop_Location_Country<br />
|-<br />
|foo:STATE||foo:Prop_Location_State<br />
|-<br />
|foo:validThrough||foo:Prop_validThrough<br />
|}<br />
<br />
In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.<br />
<br />
==Syntax Of Examples==<br />
<br />
The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].<br />
<br />
== Examples of Data Requirements ==<br />
<br />
===Example 1: Property Completeness Rule===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]<br />
|-<br />
|'''Dimension:'''||[[dqm:PropertyCompleteness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:<br />
<br />
<br />
'''Definition in OWL Full'''<br />
<pre><br />
foo:PropertyCompletenessRule_1<br />
a dqm:PropertyCompletenessRule ;<br />
dqm:testedClass http://www.example.org/MyClass ;<br />
dqm:testedProperty1 http://www.example.org/MyProperty ;<br />
dqm:requiredProperty "true"^^xsd:boolean ;<br />
dqm:requiredValue "true"^^xsd:boolean .<br />
</pre><br />
[[Data Quality Monitoring Reports#Example 1.1: Missing Properties / Values (OWL Full Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".<br />
<br />
If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:<br />
<br />
<br />
'''Definition in OWL-DL'''<br />
<pre><br />
foo:PropertyCompletenessRule_1<br />
a dqm:PropertyCompletenessRule ;<br />
dqm:testedClass foo:Class_1 ;<br />
dqm:testedProperty1 foo:Property_1 ;<br />
dqm:requiredProperty "true"^^xsd:boolean ;<br />
dqm:requiredValue "true"^^xsd:boolean .<br />
</pre><br />
[[Data Quality Monitoring Reports#Example 1.2: Missing Properties / Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".<br />
<br />
===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]<br />
|-<br />
|'''Dimension:'''||[[dqm:PropertyCompleteness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
To define this data requirement, you must perform the following two steps:<br />
<br />
'''1. Define the Condition'''<br />
<br />
<pre><br />
foo:Condition_USA<br />
a dqm:Condition ;<br />
rdfs:label "Condition USA"^^xsd:string ;<br />
dqm:conditionalProperty foo:Prop_Location_Country ;<br />
dqm:equals "USA"^^xsd:string .<br />
</pre><br />
<br />
'''2. Define the Conditional Property Completeness Rule'''<br />
<br />
<pre><br />
foo:ConditionalPropertyCompletenessRule_State<br />
a dqm:ConditionalPropertyCompletenessRule ;<br />
rdfs:label "Conditional property completeness rule State"^^xsd:string ;<br />
dqm:hasCondition1 foo:Condition_USA ;<br />
dqm:requiredProperty "true"^^xsd:boolean ;<br />
dqm:requiredValue "true"^^xsd:boolean ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_State .<br />
</pre><br />
[[Data Quality Monitoring Reports#Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 3: Syntax Rule (OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]<br />
|}<br />
<br />
You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:SyntaxRule_ZIP<br />
a dqm:SyntaxRule ;<br />
rdfs:label "Syntax rule ZIP"^^xsd:string ;<br />
dqm:regex "^[0-9]{5}$"^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Property_ZIP .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 3: Syntax Violations (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
In order to specify a conditional syntax rule, you must perform the following steps:<br />
<br />
'''1.Define Condition'''<br />
<pre><br />
foo:Condition_USA<br />
a dqm:Condition ;<br />
rdfs:label "Condition USA"^^xsd:string ;<br />
dqm:conditionalProperty foo:Prop_Location_Country ;<br />
dqm:equals "USA"^^xsd:string .<br />
</pre><br />
<br />
'''2.Define Conditional Syntax Rule'''<br />
<pre><br />
foo:ConditionalSyntaxRule_State<br />
a dqm:ConditionalSyntaxRule ;<br />
rdfs:label "Conditional syntax rule State"^^xsd:string ;<br />
dqm:hasCondition1 foo:Condition_USA ;<br />
dqm:regex "^[A-Z]{2}$"^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_State .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 5: Legal Value Range Rule (OWL DL)===<br />
{|class="wikitable"<br />
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.<br />
|-<br />
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]<br />
|}<br />
<br />
You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:<br />
<br />
<pre><br />
foo:LegalValueRangeRule_Price<br />
a dqm:LegalValueRangeRule ;<br />
rdfs:label "Legal value range rule Price"^^xsd:string ;<br />
dqm:lowerLimit "0.00"^^xsd:float ;<br />
dqm:testedClass foo:Class_Product ;<br />
dqm:testedProperty1 foo:Prop_Product_Price .<br />
</pre><br />
<br />
The class [[dqm:LegalValueRangeRule]] has the special properties<br />
* [[dqm:lowerLimit]]<br />
* [[dqm:upperLimit]]<br />
<br />
to specify the lowest and/or highest allowed value.<br />
<br />
[[Data Quality Monitoring Reports#Example 5: Out of Range Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 6: Legal Value Rule (OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify a reference property that holds the allowed values.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]<br />
|}<br />
<br />
In order to specify legal values for a specific property, you need to perform the following steps:<br />
<br />
# Create a reference data set or use an existing data set that holds the legal values <br />
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]<br />
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:LegalValueRule_Country<br />
a dqm:LegalValueRule ;<br />
rdfs:label "Legal value rule Country"^^xsd:string ;<br />
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;<br />
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_Country .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 6: Illegal Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 7: Unique Value Rule (OWL DL)===<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that values of a property must be unique.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:PropertyUniqueness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]<br />
|}<br />
<br />
You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:<br />
<br />
<pre><br />
foo:UniqueValueRule_LOCID<br />
a dqm:UniqueValueRule ;<br />
rdfs:label "Unique value rule LOCID"^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_ID .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 7: Uniqueness Violations (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.<br />
|-<br />
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SemanticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
In order to specify the dependency between two property values, you must perform the following steps:<br />
<br />
# Define a condition under which a specific value is always required. <br />
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:FuncDepValueRule_1<br />
a dqm:FuncDepValueRule ;<br />
rdfs:label "Func dep value rule 1"^^xsd:string ;<br />
dqm:equals "USA"^^xsd:string ;<br />
dqm:hasCondition1 foo:Condition_New_York ;<br />
dqm:reqDescription "If the city value is \"New York\" then the country must be <br />
\"USA\"."^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_Country .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.<br />
|-<br />
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]<br />
|-<br />
|'''Dimension:'''||[[dqm:SemanticAccuracy]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]<br />
|}<br />
<br />
In order to specify the dependency between three property values, you must perform the following steps:<br />
<br />
# Define both conditions under which a specific value is always required. <br />
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:FuncDepValueRule_2<br />
a dqm:FuncDepValueRule ;<br />
rdfs:label "Func dep value rule 2"^^xsd:string ;<br />
dqm:equals "NY"^^xsd:string ;<br />
dqm:hasCondition1 foo:Condition_USA ;<br />
dqm:hasCondition2 foo:Condition_New_York ;<br />
dqm:reqDescription "If the city value is \"New York\" and the country value is <br />
\"USA\" then the state must be \"NY\"."^^xsd:string ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_State .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 10: Expiry Rule (OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that instances of a specific class expire.<br />
|-<br />
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]<br />
|-<br />
|'''Dimension:'''||[[dqm:Timeliness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:ClassRequirement]]<br />
|}<br />
<br />
You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:ExpiryRule_1<br />
a dqm:ExpiryRule ;<br />
rdfs:label "Expiry rule 1"^^xsd:string ;<br />
dqm:testedClass foo:Class_Product ;<br />
dqm:testedProperty1 foo:Prop_Product_validThrough .<br />
</pre><br />
<br />
[[Data Quality Monitoring Reports#Example 10: Outdated / Expired Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /><br />
<br />
===Example 11: Update Rule (OWL DL)===<br />
<br />
{|class="wikitable"<br />
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.<br />
|-<br />
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.<br />
|-<br />
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]<br />
|-<br />
|'''Dimension:'''||[[dqm:Timeliness]]<br />
|-<br />
|'''Requirement Type:'''||[[dqm:ClassRequirement]]<br />
|}<br />
<br />
You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:<br />
<br />
<pre><br />
foo:UpdateRule_Location<br />
a dqm:UpdateRule ;<br />
rdfs:label "Update rule Location"^^xsd:string ;<br />
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;<br />
dqm:testedClass foo:Class_Location ;<br />
dqm:testedProperty1 foo:Prop_Location_timestamp .<br />
</pre><br />
<br />
<span style="color:red">'''NOTE:'''</span> The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.<br />
<br />
[[Data Quality Monitoring Reports#Example 11: Outdated / Not Updated Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]<br /></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T21:02:20Z<p>Admin: /* Example 10: Expired Values (OWL DL Design) */</p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br /><br />
Problem reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Outdated / Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T21:02:02Z<p>Admin: /* Example 11: Oudated Values (OWL DL Design) */</p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br /><br />
Problem reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 11: Outdated / Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T21:01:44Z<p>Admin: /* Example 11: Not Updated Values (OWL DL Design) */</p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br /><br />
Problem reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 11: Oudated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-20T20:58:59Z<p>Admin: /* Consuming the DQM-Vocabulary */</p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br />
<br />
== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support [[Data Quality]] management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent [[Data Requirements]], i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and [[Data Quality]] scores based on [[Data Requirements]]<br />
* Automated consistency checks between [[Data Requirements]]<br />
* Exchange of [[Data Quality]] information and [[Data Requirements]] on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Vocabulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Expressing Your Own Data Requirements ==<br />
* [[Create Data Requirements]]<br />
* [[Requirement Metadata]]<br />
<br />
== Consuming the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Generate Data Quality Score Reports]]<br />
<br />
== Data Quality Knowledge Management==<br />
* [[Generate Problem Annotations]]<br />
* [[Generate Score Annotations]]<br />
* [[Data Requirements Consistency Checks]]<br />
<br />
* [[Data Quality Monitoring Reports|Data Quality Monitoring Reports (<span style="color:red">'''DEPRECATED'''</span>)]]<br />
<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
<br />
== See also ==<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Quality Perception in Information Systems]]<br />
<br />
== References ==<br />
'''Fürber, Christian and Hepp, Martin''': [http://www.fuerber.com/publications/Fuerber-Hepp-Towards%20a%20vocabulary%20for%20dqm.pdf Towards a Vocabulary for Data Quality Management in Semantic Web Architectures], Proceedings of the 1st International Workshop on Linked Web Data Management (LWDM2011), Uppsala, Sweden, March 25, 2011.</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Generate_Problem_ReportsGenerate Problem Reports2011-10-20T20:58:22Z<p>Admin: moved Generate Problem Reports to Data Quality Monitoring Reports</p>
<hr />
<div>#REDIRECT [[Data Quality Monitoring Reports]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_Monitoring_ReportsData Quality Monitoring Reports2011-10-20T20:58:22Z<p>Admin: moved Generate Problem Reports to Data Quality Monitoring Reports</p>
<hr />
<div>'''<span style="color:red">This site is currently under construction!</span>'''<br /><br />
Problem reports are an excellent instrument to quickly identify instances with requirement violations.<br />
<br />
==Prerequisites==<br />
With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].<br />
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:<br />
<br />
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]<br />
* [[Create Data Requirements]]<br />
<br />
==Data Requirement Violation Reports with SPARQL==<br />
<br />
Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.<br />
<br />
<br />
===Example 1.1: Missing Properties / Values (OWL Full Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL Full Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
{ <br />
?i a ?tclass .<br />
NOT EXISTS{<br />
?i ?tprop ?value .<br />
}<br />
}UNION{<br />
?i a ?tclass .<br />
?i ?tprop "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 1.2: Missing Properties / Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:PropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
{ <br />
?i a ?tclassURI .<br />
NOT EXISTS{<br />
?i ?tpropURI ?value .<br />
}<br />
}UNION{<br />
?i a ?tclassURI .<br />
?i ?tpropURI "" .<br />
}<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.<br />
|-<br />
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||Instances with missing values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?condvalue<br />
WHERE {<br />
?dqr a dqm:ConditionalPropertyCompletenessRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:requiredProperty "true"^^xsd:boolean .<br />
?dqr dqm:requiredValue "true"^^xsd:boolean .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
{<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value <br />
NOT EXISTS{<br />
?i ?tpropURI ?value1 .<br />
}<br />
}UNION{ <br />
?i a ?tclassURI .<br />
?i ?cpropURI ?value .<br />
?i ?tpropURI "" .<br />
}<br />
FILTER (str(?value) = str(?condvalue)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 3: Syntax Violations (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.<br />
|-<br />
|'''Input:'''||All syntax rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with property values that have syntax violations<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:SyntaxRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (!regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.<br />
|-<br />
|'''Input:'''||All conditional syntax rules(in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances of the subset of the class that have syntax violations.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ConditionalSyntaxRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:regex ?regex .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop .<br />
?cprop dqm:hasURI ?cpropreal .<br />
?cond1 dqm:equals ?condvalue .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI ?condvalueplain .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 5: Out of Range Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have property values out of a specified range.<br />
|-<br />
|'''Input:'''||All legal value range rules (in OWL DL Design)<br />
|-<br />
|'''Output:'''||All instances with out of range values<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRangeRule .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:testedProperty1 ?tprop .<br />
OPTIONAL{<br />
?dqr dqm:upperLimit ?upperLimit .<br />
}<br />
OPTIONAL{<br />
?dqr dqm:lowerLimit ?lowerLimit .<br />
}<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit || <br />
STRDT(str(?value), xsd:float) < ?lowerLimit) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 6: Illegal Values (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.<br />
|-<br />
|'''Input:'''||All legal value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with values not listed in the trusted reference.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:LegalValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?dqr dqm:referenceClass ?rclass .<br />
?dqr dqm:referenceProperty1 ?rprop .<br />
?rclass dqm:hasURI ?rclassvalue .<br />
?rprop dqm:hasURI ?rpropvalue .<br />
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .<br />
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?value .<br />
OPTIONAL {<br />
?i2 a ?rclassURI .<br />
?i2 ?rpropURI ?value1 .<br />
FILTER (str(?value1) = str(?value)) .<br />
} .<br />
FILTER (!bound(?value1)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 7: Uniqueness Violations (OWL DL Design)===<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with property values that are not unique although they should.<br />
|-<br />
|'''Input:'''||All unique value rules (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UniqueValueRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?uniqueValue1 .<br />
?i2 ?tpropURI ?uniqueValue2 .<br />
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))<br />
}<br />
GROUP BY ?dqr ?i<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i ?dvalue ?value<br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
MINUS{<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all instances with inconsistent property values.<br />
|-<br />
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).<br />
|-<br />
|'''Output:'''||All instances with inconsistent property values.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i <br />
WHERE {<br />
?dqr a dqm:FuncDepValueRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?dqr dqm:equals ?dvalue .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
?dqr dqm:hasCondition1 ?cond1 .<br />
?cond1 dqm:conditionalProperty ?cprop1 .<br />
?cprop1 dqm:hasURI ?cpropreal1 .<br />
?cond1 dqm:equals ?condvalue1 .<br />
?dqr dqm:hasCondition2 ?cond2 .<br />
?cond2 dqm:conditionalProperty ?cprop2 .<br />
?cprop2 dqm:hasURI ?cpropreal2 .<br />
?cond2 dqm:equals ?condvalue2 .<br />
MINUS{<br />
?dqr dqm:hasCondition3 ?cond3 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition4 ?cond4 .<br />
}<br />
MINUS{<br />
?dqr dqm:hasCondition5 ?cond5 .<br />
}<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .<br />
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .<br />
?i a ?tclassURI .<br />
?i ?cpropURI1 ?condvalue1 .<br />
?i ?cpropURI2 ?condvalue2 .<br />
?i ?tpropURI ?value .<br />
FILTER (str(?dvalue) != str(?value)) .<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
===Example 10: Expired Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.<br />
|-<br />
|'''Output:'''||All instances with that are not valid anymore.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:ExpiryRule .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:testedClass ?tclass . <br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND ((now()) AS ?today)<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?expiry .<br />
FILTER(?expiry<?today)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]<br />
<br />
<br />
===Example 11: Not Updated Values (OWL DL Design)===<br />
<br />
{|class="wikitable"<br />
|'''Scenario:'''||Find all outdated instances.<br />
|-<br />
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.<br />
|-<br />
|'''Output:'''||All instances with that are potentially outdated.<br />
|}<br />
<syntaxhighlight lang="sparql"><br />
SELECT ?dqr ?i<br />
WHERE {<br />
?dqr a dqm:UpdateRule .<br />
?dqr dqm:testedClass ?tclass .<br />
?dqr dqm:testedProperty1 ?tprop .<br />
?dqr dqm:expectedUpdateInterval ?duration .<br />
?tclass dqm:hasURI ?tclassreal .<br />
?tprop dqm:hasURI ?tpropreal .<br />
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .<br />
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .<br />
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .<br />
?i a ?tclassURI .<br />
?i ?tpropURI ?timestamp .<br />
FILTER(?reqTimestamp>?timestamp)<br />
}<br />
</syntaxhighlight><br />
<br />
[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-09T14:30:18Z<p>Admin: /* Purpose and Usage of the DQM-Vocabulary */</p>
<hr />
<div>== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support [[Data Quality]] management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent [[Data Requirements]], i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and [[Data Quality]] scores based on [[Data Requirements]]<br />
* Automated consistency checks between [[Data Requirements]]<br />
* Exchange of [[Data Quality]] information and [[Data Requirements]] on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Vocabulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Quick Start ==<br />
<br />
* [[Create Data Requirements]]<br />
* [[Generate Problem and Score Annotations]]<br />
* [[Generate Problem Reports]]<br />
* [[Generate Data Quality Score Reports]]<br />
<br />
== Examples ==<br />
* [[Creating Data Requirements]]<br />
* [[Usage of DQM-Vocab as OWL DL or OWL Full]]<br />
<br />
== Querying the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== See also ==<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-09T13:41:25Z<p>Admin: /* Purpose and Usage of the DQM-Vocabulary */</p>
<hr />
<div>== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support [[Data Quality]] management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent [[Data Requirements]], i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and [[Data Quality]] scores based on [[Data Requirements]]<br />
* Automated consistency checks between [[Data Requirements]]<br />
* Exchange of [[Data Quality]] information and [[Data Requirements]] on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Voacbulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Quick Start ==<br />
<br />
* [[Create Data Requirements]]<br />
* [[Generate Problem and Score Annotations]]<br />
* [[Generate Problem Reports]]<br />
* [[Generate Data Quality Score Reports]]<br />
<br />
== Examples ==<br />
* [[Creating Data Requirements]]<br />
* [[Usage of DQM-Vocab as OWL DL or OWL Full]]<br />
<br />
== Querying the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== See also ==<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-09T13:40:07Z<p>Admin: /* Purpose and Usage of the DQM-Vocabulary */</p>
<hr />
<div>== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support data quality management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent data requirements, i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and data quality scores based on [[data requirements]]<br />
* Automated consistency checks between data requirements<br />
* Exchange of data quality information and data requirements on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Voacbulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Quick Start ==<br />
<br />
* [[Create Data Requirements]]<br />
* [[Generate Problem and Score Annotations]]<br />
* [[Generate Problem Reports]]<br />
* [[Generate Data Quality Score Reports]]<br />
<br />
== Examples ==<br />
* [[Creating Data Requirements]]<br />
* [[Usage of DQM-Vocab as OWL DL or OWL Full]]<br />
<br />
== Querying the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== See also ==<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-09T13:39:35Z<p>Admin: /* Purpose and Usage of the DQM-Vocabulary */</p>
<hr />
<div>== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support data quality management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent data requirements, i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and data quality scores based on data requirements<br />
* Automated consistency checks between data requirements<br />
* Exchange of data quality information and data requirements on web-scale<br />
<br /><br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Voacbulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Quick Start ==<br />
<br />
* [[Create Data Requirements]]<br />
* [[Generate Problem and Score Annotations]]<br />
* [[Generate Problem Reports]]<br />
* [[Generate Data Quality Score Reports]]<br />
<br />
== Examples ==<br />
* [[Creating Data Requirements]]<br />
* [[Usage of DQM-Vocab as OWL DL or OWL Full]]<br />
<br />
== Querying the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== See also ==<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-09T13:39:22Z<p>Admin: /* Purpose and Usage of the DQM-Vocabulary */</p>
<hr />
<div>== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support data quality management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent data requirements, i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and data quality scores based on data requirements<br />
* Automated consistency checks between data requirements<br />
* Exchange of data quality information and data requirements on web-scale<br />
<br />
:::'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Voacbulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Quick Start ==<br />
<br />
* [[Create Data Requirements]]<br />
* [[Generate Problem and Score Annotations]]<br />
* [[Generate Problem Reports]]<br />
* [[Generate Data Quality Score Reports]]<br />
<br />
== Examples ==<br />
* [[Creating Data Requirements]]<br />
* [[Usage of DQM-Vocab as OWL DL or OWL Full]]<br />
<br />
== Querying the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== See also ==<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-09T13:39:00Z<p>Admin: /* Purpose and Usage of the DQM-Vocabulary */</p>
<hr />
<div>== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support data quality management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent data requirements, i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and data quality scores based on data requirements<br />
* Automated consistency checks between data requirements<br />
* Exchange of data quality information and data requirements on web-scale<br />
<br />
'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Voacbulary]'''''<br /><br />
<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Quick Start ==<br />
<br />
* [[Create Data Requirements]]<br />
* [[Generate Problem and Score Annotations]]<br />
* [[Generate Problem Reports]]<br />
* [[Generate Data Quality Score Reports]]<br />
<br />
== Examples ==<br />
* [[Creating Data Requirements]]<br />
* [[Usage of DQM-Vocab as OWL DL or OWL Full]]<br />
<br />
== Querying the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== See also ==<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=DQM-Vocabulary_PrimerDQM-Vocabulary Primer2011-10-09T13:38:46Z<p>Admin: /* Purpose and Usage of the DQM-Vocabulary */</p>
<hr />
<div>== Purpose and Usage of the DQM-Vocabulary ==<br />
The DQM-Vocabulary was created to support data quality management acitivities in [[Wikipedia:Semantic Web|Semantic Web]] architectures. It's major stregth is the ability to represent data requirements, i.e. quality-relevant expectations on data, so that computers can interpret the requirements and take further actions. Among other things, the DQM-Vocabulary enables the following core-features:<br />
<br />
* Automated creation of problem reports and data quality scores based on data requirements<br />
* Automated consistency checks between data requirements<br />
* Exchange of data quality information and data requirements on web-scale<br />
<br />
'''''[http://www.slideshare.net/cfuerber/towards-a-vocabulary-for-data-quality-management-in-semantic-web-architectures Slideshare Presentation on DQM-Voacbulary]'''''<br />
{{#widget:SlideShare<br />
|doc=dqm-vocabularyv1-0-110325125708-phpapp02<br />
|width=425<br />
|height=348<br />
}}<br />
<br />
== Quick Start ==<br />
<br />
* [[Create Data Requirements]]<br />
* [[Generate Problem and Score Annotations]]<br />
* [[Generate Problem Reports]]<br />
* [[Generate Data Quality Score Reports]]<br />
<br />
== Examples ==<br />
* [[Creating Data Requirements]]<br />
* [[Usage of DQM-Vocab as OWL DL or OWL Full]]<br />
<br />
== Querying the DQM-Vocabulary ==<br />
* [[Data Quality Monitoring Reports]]<br />
* [[Data Quality Assessment Reports]]<br />
<br />
== See also ==<br />
== Data Quality Management Vocabulary ==<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-10-09T13:19:01Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]]<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Quality Perception in Information Systems]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td colspan="3"><br />
{{BigHeading<br />
|Title=Links to Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_QualityData Quality2011-10-09T13:17:29Z<p>Admin: </p>
<hr />
<div>There are multiple different ways to define data quality and there is currently no commonly agreed definition on what data quality is. However, the following table provides an overview of popular data quality definitions. <br />
<br />
{| class="wikitable sortable"<br />
|+ Popular Data Quality Definitions<br />
|-<br />
! Authors !! Data Quality Definition<br />
|-<br />
| Wang and Strong (1996) || “[…] data that are fit for use by data consumers.”<ref>Wang, R. Y., & Strong, D. M. (1996). Beyond accuracy: what data quality means to data consumers. Journal of Management Information Systems, 12(4), 5-33.</ref><br />
|-<br />
| Kahn, Strong, and Wang (2002)|| “conformance to specifications” and “meeting or exceeding consumer expectations”<ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref><br />
|-<br />
| Redman (2001)|| “Data are of high quality if they are fit for their intended uses in operations, decision making, and planning. Data are fit for use if they are free of defects and possess desired features.”<ref>Redman, T. C. (2001). Data quality: the field guide. Boston: Digital Press.</ref><br />
|-<br />
| Olson (2003) || “[…] data has quality if it satisfies the requirements of its intended use.”<ref>Olson, J. (2003). Data quality: the accuracy dimension. San Francisco, USA: Morgan Kaufmann; Elsevier Science.</ref><br />
|-<br />
|ISO 8000 || Quality is the "degree to which a set of inherent characteristics fulfils requirements"<ref>ISO (2005) ISO8000-102:2009, Data quality — Part 102: Master data: Exchange of characteristic data: Vocabulary</ref><br />
|}<br />
<br />
All of the above definitions have something in common: data quality encompasses the comparison of the '''status quo''' of data to its '''desired state'''. The desired state has multiple different names, such as "fitness for use", "specification", "consumer expectations", "defect-free", "desired features", or simply "requirements". The desired state may thereby not only be stated by data consumers, but also by data providers, administrators, legal authorities, and many other stakeholders. Thus, there are multiple different perspectives on requirements. Based on this analysis, we can derive the following definition of data quality:<br />
<br />
'''Data quality is the degree to which requirements are fulfilled'''. <br />
<br />
The requirements can thereby be manyfold due to multiple different tastes, needs, and perspectives. Hence, data quality is also multi-dimensional. Wang and Strong identified the following 15 most important dimensions in the eyes of data consumers during an empirical study in 1996 <ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref>:<br />
<br />
{| class="wikitable"<br />
|+The 15 most important data quality dimensions from consumer perspective<ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref><br />
|-<br />
! Category !! Dimension !! Definition<br />
|-<br />
|rowspan="4"| Intrinsic || Believability || “The extent to which data are accepted or regarded as true, real and credible.”<br />
|-<br />
|| Accuracy || “The extent to which data are correct, reliable and certified free of error.”<br />
|-<br />
|| Objectivity || “The extent to which data are unbiased (unprejudiced) and impartial.”<br />
|-<br />
|| Reputation || “The extent to which data are trusted or highly regarded in terms of their source or content.”<br />
|-<br />
|rowspan="5"| Contextual|| Value-added|| “The extent to which data are beneficial and provide advantages from their use.”<br />
|-<br />
|| Relevancy|| “The extent to which data are applicable and helpful for the task at hand.”<br />
|-<br />
|| Timeliness|| “The extent to which the age of the data is appropriate for the task at hand.”<br />
|-<br />
|| Completeness|| “The extent to which data are of sufficient depth, breadth, and scope for the task at hand.”<br />
|-<br />
|| Appropriate amount of data|| “The extent to which the quantity and volume of available data is appropriate.”<br />
|-<br />
| rowspan="4"|Representational|| Interpretability || “The extent to which data are in appropriate language and units and the data definitions are clear.”<br />
|-<br />
|| Ease of understanding|| “The extent to which data are clear without ambiguity and easily comprehended.”<br />
|-<br />
|| Representational consistency|| “The extent to which data are always presented in the same format and are compatible with previous data.”<br />
|-<br />
|| Concise representation || “The extent to which data are compactly represented without being overwhelming (i.e., brief in presentation, yet complete and to the point).”<br />
|-<br />
| rowspan="2"|Accessibility|| Accessibility|| “The extent to which data are available or easily and quickly retrievable.”<br />
|-<br />
|| Access security|| “The extent to which access to data can be restricted and hence kept secure.”<br />
|}<br />
<br />
==See Also==<br />
[[Quality Perception in Information Systems]]<br /><br />
[[Data Requirements]]<br /><br />
[http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br /><br />
<br />
----<br />
<references /><br />
<br />
''(This Article was created by Christian Fürber on October 09th 2011)''</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-10-09T13:17:04Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]]<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Data Requirements]]<br />
* [[Quality Perception in Information Systems]]<br />
* [[Data Quality Management Methodologies]]<br />
* [[Core Problems of Data Quality Management]]<br />
* [[Data Quality Problem Typology]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td colspan="3"><br />
{{BigHeading<br />
|Title=Links to Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-10-09T13:16:31Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]]<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Quality Perception in Information Systems]]<br />
* [[Data Quality Management Methodologies]]<br />
* [[Core Problems of Data Quality Management]]<br />
* [[Data Quality Problem Typology]]<br />
* [[Data Requirements]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td colspan="3"><br />
{{BigHeading<br />
|Title=Links to Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_RequirementsData Requirements2011-10-09T13:15:13Z<p>Admin: </p>
<hr />
<div>A data requirement is a prescribed directive or consensual agreement that defines the content and/or structure that constitute high quality data instances and values. The following table contains different types of data requirements that can be expressed via the [http://semwebquality.org/ontologies/dq-constraints# DQ-Constraint Library] or the [http://purl.org/dqm-vocabulary/v1/dqm# DQM-Vocabulary].<br />
<br />
<br />
{|class="wikitable sortable"<br />
|+ Types of Data Requirements<br />
|-<br />
!Data Quality Rule <br />
!Definition <br />
!Example <br />
![http://semwebquality.org/ontologies/dq-constraints# DQ-Constraint Lib] for [http://spinrdf.org SPIN]<br />
![http://purl.org/dqm-vocabulary/v1/dqm# DQM-Vocabulary]<br />
|-<br />
|'''Mandatory Property and Literal Rules''' <br />
|Properties and their literals become mandatory, if the data is required for the task at hand. <br />
|The properties indicating the geographical coordinates must exist and have values for all instances of the class foo:Location to be able to navigate to each location. <br />
|[[dqc:MissingElementConstraints]]<br />
[[dqc:FunctionalDependentEmptyValueAll]]<br />
|[[dqm:MissingPropertyRule]]<br />
[[dqm:MissingValueRule]]<br />
[[dqm:ConditionalPropertyRule]]<br />
[[dqm:ConditionalValueRule]]<br />
|-<br />
|'''Syntactic Rules ''' <br />
|Syntactic rules define the type of characters and/or the pattern of literal values. <br />
|Literal values for the property foo:country-name must only contain letters. <br />
|[[dqc:SyntaxConstraints]]<br />
|[[dqm:SyntaxRule]]<br />
[[dqm:ConditionalSyntaxRule]]<br />
|-<br />
|'''Functional Dependency Rules''' <br />
|Functional dependencies are dependencies between the values of two or more different properties. <br />
|The literal value for foo:city is always dependent to the literal value for foo:country, since certain city names only exist in certain countries.<br />
|[[dqc:FunctionalDependencyConstraints]]<br />
|[[dqm:FuncDepValueRule]]<br />
[[dqm:FuncDepReferenceRule]]<br />
[[dqm:CustomRequirement]]<br />
|-<br />
|'''Legal Value Rules''' <br />
|Legal value rules are the explicit definition of the allowed values for a certain property. <br />
|The property foo:gender must only contain the values “male”, “female”, “m”, or “f”. <br />
|[[dqc:LegalValueConstraints]]<br />
|[[dqm:LegalValueRule]]<br />
|-<br />
|'''Legal Value Range Rules''' <br />
|Legal value range rules are the explicit definition of the allowed value range for properties holding numerical values. A value range contains an upper and/or lower limit. <br />
|The property foo:population must only contain values greater than zero.<br />
|[[dqc:ValueRangeConstraints]]<br />
|[[dqm:LegalValueRangeRule]]<br />
|-<br />
|'''Illegal Value Rules''' <br />
|Illegal value rules are the explicit definition of the values that may not be assigned to a certain property. <br />
|The property foo:gender may never contain the value “mail”. <br />
|[[dqc:LegalValueConstraints]]<br />
|[[dqm:IllegalValueRule]]<br />
|-<br />
|'''Illegal Value Range Rules''' <br />
|Illegal value range rules are the explicit definition of prohibited value ranges for properties holding numerical values. A value range contains an upper and/or lower limit. <br />
|The property foo:population must not contain values less than one.<br />
|[[dqc:ValueRangeConstraints]]<br />
|[[dqm:IllegalValueRangeRule]]<br />
|-<br />
|'''Unique Value Rules'''<br />
|Unique value rules define properties that may contain each literal value not more than once within a defined collection of values. <br />
|Each value for property foo:ISBN in instances of class foo:Book may not occur more than once. <br />
|[[dqc:UniquenessConstraints]]<br />
|[[dqm:UniqueValueRule]]<br />
|-<br />
|'''Outdated Value Rules'''<br />
|Outdated value rules are rules that identify instances that represent an outdated state of the corresponding real world entity. <br />
|Instances of the class foo:Offer are outdated, if its value for foo:validThrough is elder than the current date and time. <br />
|Not covered yet<br />
|[[dqm:UpdateRule]]<br />
[[dqm:ExpiryRule]]<br />
|-<br />
|'''Duplicate Instance Rules'''<br />
|<br />
|<br />
|Not covered yet<br />
|[[dqm:DuplicateInstanceRule]]<br />
|}</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_Quality_RulesData Quality Rules2011-10-09T13:12:45Z<p>Admin: moved Data Quality Rules to Data Requirements</p>
<hr />
<div>#REDIRECT [[Data Requirements]]</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_RequirementsData Requirements2011-10-09T13:12:45Z<p>Admin: moved Data Quality Rules to Data Requirements</p>
<hr />
<div>{|class="wikitable sortable"<br />
!Data Quality Rule <br />
!Definition <br />
!Example <br />
![http://semwebquality.org/ontologies/dq-constraints# DQ-Constraint Lib] for [http://spinrdf.org SPIN]<br />
![http://purl.org/dqm-vocabulary/v1/dqm# DQM-Vocabulary]<br />
|-<br />
|'''Mandatory Property and Literal Rules''' <br />
|Properties and their literals become mandatory, if the data is required for the task at hand. <br />
|The properties indicating the geographical coordinates must exist and have values for all instances of the class foo:Location to be able to navigate to each location. <br />
|[[dqc:MissingElementConstraints]]<br />
[[dqc:FunctionalDependentEmptyValueAll]]<br />
|[[dqm:MissingPropertyRule]]<br />
[[dqm:MissingValueRule]]<br />
[[dqm:ConditionalPropertyRule]]<br />
[[dqm:ConditionalValueRule]]<br />
|-<br />
|'''Syntactic Rules ''' <br />
|Syntactic rules define the type of characters and/or the pattern of literal values. <br />
|Literal values for the property foo:country-name must only contain letters. <br />
|[[dqc:SyntaxConstraints]]<br />
|[[dqm:SyntaxRule]]<br />
[[dqm:ConditionalSyntaxRule]]<br />
|-<br />
|'''Functional Dependency Rules''' <br />
|Functional dependencies are dependencies between the values of two or more different properties. <br />
|The literal value for foo:city is always dependent to the literal value for foo:country, since certain city names only exist in certain countries.<br />
|[[dqc:FunctionalDependencyConstraints]]<br />
|[[dqm:FuncDepValueRule]]<br />
[[dqm:FuncDepReferenceRule]]<br />
[[dqm:CustomRequirement]]<br />
|-<br />
|'''Legal Value Rules''' <br />
|Legal value rules are the explicit definition of the allowed values for a certain property. <br />
|The property foo:gender must only contain the values “male”, “female”, “m”, or “f”. <br />
|[[dqc:LegalValueConstraints]]<br />
|[[dqm:LegalValueRule]]<br />
|-<br />
|'''Legal Value Range Rules''' <br />
|Legal value range rules are the explicit definition of the allowed value range for properties holding numerical values. A value range contains an upper and/or lower limit. <br />
|The property foo:population must only contain values greater than zero.<br />
|[[dqc:ValueRangeConstraints]]<br />
|[[dqm:LegalValueRangeRule]]<br />
|-<br />
|'''Illegal Value Rules''' <br />
|Illegal value rules are the explicit definition of the values that may not be assigned to a certain property. <br />
|The property foo:gender may never contain the value “mail”. <br />
|[[dqc:LegalValueConstraints]]<br />
|[[dqm:IllegalValueRule]]<br />
|-<br />
|'''Illegal Value Range Rules''' <br />
|Illegal value range rules are the explicit definition of prohibited value ranges for properties holding numerical values. A value range contains an upper and/or lower limit. <br />
|The property foo:population must not contain values less than one.<br />
|[[dqc:ValueRangeConstraints]]<br />
|[[dqm:IllegalValueRangeRule]]<br />
|-<br />
|'''Unique Value Rules'''<br />
|Unique value rules define properties that may contain each literal value not more than once within a defined collection of values. <br />
|Each value for property foo:ISBN in instances of class foo:Book may not occur more than once. <br />
|[[dqc:UniquenessConstraints]]<br />
|[[dqm:UniqueValueRule]]<br />
|-<br />
|'''Outdated Value Rules'''<br />
|Outdated value rules are rules that identify instances that represent an outdated state of the corresponding real world entity. <br />
|Instances of the class foo:Offer are outdated, if its value for foo:validThrough is elder than the current date and time. <br />
|Not covered yet<br />
|[[dqm:UpdateRule]]<br />
[[dqm:ExpiryRule]]<br />
|-<br />
|'''Duplicate Instance Rules'''<br />
|<br />
|<br />
|Not covered yet<br />
|[[dqm:DuplicateInstanceRule]]<br />
|}</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_QualityData Quality2011-10-09T13:11:17Z<p>Admin: /* See Also */</p>
<hr />
<div>There are multiple different ways to define data quality and there is currently no commonly agreed definition on what data quality is. However, the following table provides an overview of popular data quality definitions. <br />
<br />
{| class="wikitable sortable"<br />
|+ Popular Data Quality Definitions<br />
|-<br />
! Authors !! Data Quality Definition<br />
|-<br />
| Wang and Strong (1996) || “[…] data that are fit for use by data consumers.”<ref>Wang, R. Y., & Strong, D. M. (1996). Beyond accuracy: what data quality means to data consumers. Journal of Management Information Systems, 12(4), 5-33.</ref><br />
|-<br />
| Kahn, Strong, and Wang (2002)|| “conformance to specifications” and “meeting or exceeding consumer expectations”<ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref><br />
|-<br />
| Redman (2001)|| “Data are of high quality if they are fit for their intended uses in operations, decision making, and planning. Data are fit for use if they are free of defects and possess desired features.”<ref>Redman, T. C. (2001). Data quality: the field guide. Boston: Digital Press.</ref><br />
|-<br />
| Olson (2003) || “[…] data has quality if it satisfies the requirements of its intended use.”<ref>Olson, J. (2003). Data quality: the accuracy dimension. San Francisco, USA: Morgan Kaufmann; Elsevier Science.</ref><br />
|-<br />
|ISO 8000 || Quality is the "degree to which a set of inherent characteristics fulfils requirements"<ref>ISO (2005) ISO8000-102:2009, Data quality — Part 102: Master data: Exchange of characteristic data: Vocabulary</ref><br />
|}<br />
<br />
All of the above definitions have something in common: data quality encompasses the comparison of the '''status quo''' of data to its '''desired state'''. The desired state has multiple different names, such as "fitness for use", "specification", "consumer expectations", "defect-free", "desired features", or simply "requirements". The desired state may thereby not only be stated by data consumers, but also by data providers, administrators, legal authorities, and many other stakeholders. Thus, there are multiple different perspectives on requirements. Based on this analysis, we can derive the following definition of data quality:<br />
<br />
'''Data quality is the degree to which requirements are fulfilled'''. <br />
<br />
The requirements can thereby be manyfold due to multiple different tastes, needs, and perspectives. Hence, data quality is also multi-dimensional. Wang and Strong identified the following 15 most important dimensions in the eyes of data consumers during an empirical study in 1996 <ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref>:<br />
<br />
{| class="wikitable"<br />
|+The 15 most important data quality dimensions from consumer perspective<ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref><br />
|-<br />
! Category !! Dimension !! Definition<br />
|-<br />
|rowspan="4"| Intrinsic || Believability || “The extent to which data are accepted or regarded as true, real and credible.”<br />
|-<br />
|| Accuracy || “The extent to which data are correct, reliable and certified free of error.”<br />
|-<br />
|| Objectivity || “The extent to which data are unbiased (unprejudiced) and impartial.”<br />
|-<br />
|| Reputation || “The extent to which data are trusted or highly regarded in terms of their source or content.”<br />
|-<br />
|rowspan="5"| Contextual|| Value-added|| “The extent to which data are beneficial and provide advantages from their use.”<br />
|-<br />
|| Relevancy|| “The extent to which data are applicable and helpful for the task at hand.”<br />
|-<br />
|| Timeliness|| “The extent to which the age of the data is appropriate for the task at hand.”<br />
|-<br />
|| Completeness|| “The extent to which data are of sufficient depth, breadth, and scope for the task at hand.”<br />
|-<br />
|| Appropriate amount of data|| “The extent to which the quantity and volume of available data is appropriate.”<br />
|-<br />
| rowspan="4"|Representational|| Interpretability || “The extent to which data are in appropriate language and units and the data definitions are clear.”<br />
|-<br />
|| Ease of understanding|| “The extent to which data are clear without ambiguity and easily comprehended.”<br />
|-<br />
|| Representational consistency|| “The extent to which data are always presented in the same format and are compatible with previous data.”<br />
|-<br />
|| Concise representation || “The extent to which data are compactly represented without being overwhelming (i.e., brief in presentation, yet complete and to the point).”<br />
|-<br />
| rowspan="2"|Accessibility|| Accessibility|| “The extent to which data are available or easily and quickly retrievable.”<br />
|-<br />
|| Access security|| “The extent to which access to data can be restricted and hence kept secure.”<br />
|}<br />
<br />
==See Also==<br />
[[Quality Perception in Information Systems]]<br /><br />
[http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br /><br />
<br />
----<br />
<references /><br />
<br />
''(This Article was created by Christian Fürber on October 09th 2011)''</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_QualityData Quality2011-10-09T13:10:54Z<p>Admin: /* See Also */</p>
<hr />
<div>There are multiple different ways to define data quality and there is currently no commonly agreed definition on what data quality is. However, the following table provides an overview of popular data quality definitions. <br />
<br />
{| class="wikitable sortable"<br />
|+ Popular Data Quality Definitions<br />
|-<br />
! Authors !! Data Quality Definition<br />
|-<br />
| Wang and Strong (1996) || “[…] data that are fit for use by data consumers.”<ref>Wang, R. Y., & Strong, D. M. (1996). Beyond accuracy: what data quality means to data consumers. Journal of Management Information Systems, 12(4), 5-33.</ref><br />
|-<br />
| Kahn, Strong, and Wang (2002)|| “conformance to specifications” and “meeting or exceeding consumer expectations”<ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref><br />
|-<br />
| Redman (2001)|| “Data are of high quality if they are fit for their intended uses in operations, decision making, and planning. Data are fit for use if they are free of defects and possess desired features.”<ref>Redman, T. C. (2001). Data quality: the field guide. Boston: Digital Press.</ref><br />
|-<br />
| Olson (2003) || “[…] data has quality if it satisfies the requirements of its intended use.”<ref>Olson, J. (2003). Data quality: the accuracy dimension. San Francisco, USA: Morgan Kaufmann; Elsevier Science.</ref><br />
|-<br />
|ISO 8000 || Quality is the "degree to which a set of inherent characteristics fulfils requirements"<ref>ISO (2005) ISO8000-102:2009, Data quality — Part 102: Master data: Exchange of characteristic data: Vocabulary</ref><br />
|}<br />
<br />
All of the above definitions have something in common: data quality encompasses the comparison of the '''status quo''' of data to its '''desired state'''. The desired state has multiple different names, such as "fitness for use", "specification", "consumer expectations", "defect-free", "desired features", or simply "requirements". The desired state may thereby not only be stated by data consumers, but also by data providers, administrators, legal authorities, and many other stakeholders. Thus, there are multiple different perspectives on requirements. Based on this analysis, we can derive the following definition of data quality:<br />
<br />
'''Data quality is the degree to which requirements are fulfilled'''. <br />
<br />
The requirements can thereby be manyfold due to multiple different tastes, needs, and perspectives. Hence, data quality is also multi-dimensional. Wang and Strong identified the following 15 most important dimensions in the eyes of data consumers during an empirical study in 1996 <ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref>:<br />
<br />
{| class="wikitable"<br />
|+The 15 most important data quality dimensions from consumer perspective<ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref><br />
|-<br />
! Category !! Dimension !! Definition<br />
|-<br />
|rowspan="4"| Intrinsic || Believability || “The extent to which data are accepted or regarded as true, real and credible.”<br />
|-<br />
|| Accuracy || “The extent to which data are correct, reliable and certified free of error.”<br />
|-<br />
|| Objectivity || “The extent to which data are unbiased (unprejudiced) and impartial.”<br />
|-<br />
|| Reputation || “The extent to which data are trusted or highly regarded in terms of their source or content.”<br />
|-<br />
|rowspan="5"| Contextual|| Value-added|| “The extent to which data are beneficial and provide advantages from their use.”<br />
|-<br />
|| Relevancy|| “The extent to which data are applicable and helpful for the task at hand.”<br />
|-<br />
|| Timeliness|| “The extent to which the age of the data is appropriate for the task at hand.”<br />
|-<br />
|| Completeness|| “The extent to which data are of sufficient depth, breadth, and scope for the task at hand.”<br />
|-<br />
|| Appropriate amount of data|| “The extent to which the quantity and volume of available data is appropriate.”<br />
|-<br />
| rowspan="4"|Representational|| Interpretability || “The extent to which data are in appropriate language and units and the data definitions are clear.”<br />
|-<br />
|| Ease of understanding|| “The extent to which data are clear without ambiguity and easily comprehended.”<br />
|-<br />
|| Representational consistency|| “The extent to which data are always presented in the same format and are compatible with previous data.”<br />
|-<br />
|| Concise representation || “The extent to which data are compactly represented without being overwhelming (i.e., brief in presentation, yet complete and to the point).”<br />
|-<br />
| rowspan="2"|Accessibility|| Accessibility|| “The extent to which data are available or easily and quickly retrievable.”<br />
|-<br />
|| Access security|| “The extent to which access to data can be restricted and hence kept secure.”<br />
|}<br />
<br />
==See Also==<br />
[[Quality Perception in Information Systems]]<br />
[http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
<br />
----<br />
<references /><br />
''(This Article was created by Christian Fürber on October 09th 2011)''</div>Adminhttp://semwebquality.org/mediawiki/index.php?title=SemWebQuality.orgSemWebQuality.org2011-10-09T13:10:34Z<p>Admin: </p>
<hr />
<div>__NOTOC__<br />
{{DISPLAYTITLE:<span style="display:none">{{FULLPAGENAME}}</span>}}<br />
'''Welcome to SemWebQuality.org''' <br />
This website provides information and tools about data quality in Semantic Web architectures. <br />
<Table cellpadding="3"><br />
<tr><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}<br />
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.<br />
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation]<br /><br />
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF]<br /><br />
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]<br />
</td><br />
<td Width="50%"><br />
{{BigHeading<br />
|Title=Data Quality Management Vocabulary }}<br />
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing.<br /><br />
* [[DQM-Vocabulary Primer]]<br />
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]<br />
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]<br />
</td><br />
</tr><br />
<tr><br />
<td><br />
{{BigHeading<br />
|Title=Information around Data Quality }}<br />
* What is [[Data Quality]]?<br />
* [[Quality Perception in Information Systems]]<br />
* [[Data Quality Management Methodologies]]<br />
* [[Common Data Quality Management Problems]]<br />
* [[Data Requirements]]<br />
* [[Data Quality Rules]]<br />
* [http://www.dqglossary.com/ Data Quality Glossary]<br />
</td><br />
<td><br />
{{BigHeading<br />
|Title= Semantic Web Quality Links}}<br />
* [http://pedantic-web.org/ Pedantic Web Group]<br /><br />
* [http://qualitywebdata.org/ Qualitywebdata.org]<br /><br />
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]<br />
</td><br />
</tr><br />
<tr><br />
<td colspan="3"><br />
{{BigHeading<br />
|Title=Links to Data Quality Discussions }}<br />
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes<br />
</td><br />
</tr><br />
</Table></div>Adminhttp://semwebquality.org/mediawiki/index.php?title=Data_QualityData Quality2011-10-09T13:09:59Z<p>Admin: </p>
<hr />
<div>There are multiple different ways to define data quality and there is currently no commonly agreed definition on what data quality is. However, the following table provides an overview of popular data quality definitions. <br />
<br />
{| class="wikitable sortable"<br />
|+ Popular Data Quality Definitions<br />
|-<br />
! Authors !! Data Quality Definition<br />
|-<br />
| Wang and Strong (1996) || “[…] data that are fit for use by data consumers.”<ref>Wang, R. Y., & Strong, D. M. (1996). Beyond accuracy: what data quality means to data consumers. Journal of Management Information Systems, 12(4), 5-33.</ref><br />
|-<br />
| Kahn, Strong, and Wang (2002)|| “conformance to specifications” and “meeting or exceeding consumer expectations”<ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref><br />
|-<br />
| Redman (2001)|| “Data are of high quality if they are fit for their intended uses in operations, decision making, and planning. Data are fit for use if they are free of defects and possess desired features.”<ref>Redman, T. C. (2001). Data quality: the field guide. Boston: Digital Press.</ref><br />
|-<br />
| Olson (2003) || “[…] data has quality if it satisfies the requirements of its intended use.”<ref>Olson, J. (2003). Data quality: the accuracy dimension. San Francisco, USA: Morgan Kaufmann; Elsevier Science.</ref><br />
|-<br />
|ISO 8000 || Quality is the "degree to which a set of inherent characteristics fulfils requirements"<ref>ISO (2005) ISO8000-102:2009, Data quality — Part 102: Master data: Exchange of characteristic data: Vocabulary</ref><br />
|}<br />
<br />
All of the above definitions have something in common: data quality encompasses the comparison of the '''status quo''' of data to its '''desired state'''. The desired state has multiple different names, such as "fitness for use", "specification", "consumer expectations", "defect-free", "desired features", or simply "requirements". The desired state may thereby not only be stated by data consumers, but also by data providers, administrators, legal authorities, and many other stakeholders. Thus, there are multiple different perspectives on requirements. Based on this analysis, we can derive the following definition of data quality:<br />
<br />
'''Data quality is the degree to which requirements are fulfilled'''. <br />
<br />
The requirements can thereby be manyfold due to multiple different tastes, needs, and perspectives. Hence, data quality is also multi-dimensional. Wang and Strong identified the following 15 most important dimensions in the eyes of data consumers during an empirical study in 1996 <ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref>:<br />
<br />
{| class="wikitable"<br />
|+The 15 most important data quality dimensions from consumer perspective<ref>Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: product and service performance. Commun. ACM, 45(4), 184-192.</ref><br />
|-<br />
! Category !! Dimension !! Definition<br />
|-<br />
|rowspan="4"| Intrinsic || Believability || “The extent to which data are accepted or regarded as true, real and credible.”<br />
|-<br />
|| Accuracy || “The extent to which data are correct, reliable and certified free of error.”<br />
|-<br />
|| Objectivity || “The extent to which data are unbiased (unprejudiced) and impartial.”<br />
|-<br />
|| Reputation || “The extent to which data are trusted or highly regarded in terms of their source or content.”<br />
|-<br />
|rowspan="5"| Contextual|| Value-added|| “The extent to which data are beneficial and provide advantages from their use.”<br />
|-<br />
|| Relevancy|| “The extent to which data are applicable and helpful for the task at hand.”<br />
|-<br />
|| Timeliness|| “The extent to which the age of the data is appropriate for the task at hand.”<br />
|-<br />
|| Completeness|| “The extent to which data are of sufficient depth, breadth, and scope for the task at hand.”<br />
|-<br />
|| Appropriate amount of data|| “The extent to which the quantity and volume of available data is appropriate.”<br />
|-<br />
| rowspan="4"|Representational|| Interpretability || “The extent to which data are in appropriate language and units and the data definitions are clear.”<br />
|-<br />
|| Ease of understanding|| “The extent to which data are clear without ambiguity and easily comprehended.”<br />
|-<br />
|| Representational consistency|| “The extent to which data are always presented in the same format and are compatible with previous data.”<br />
|-<br />
|| Concise representation || “The extent to which data are compactly represented without being overwhelming (i.e., brief in presentation, yet complete and to the point).”<br />
|-<br />
| rowspan="2"|Accessibility|| Accessibility|| “The extent to which data are available or easily and quickly retrievable.”<br />
|-<br />
|| Access security|| “The extent to which access to data can be restricted and hence kept secure.”<br />
|}<br />
<br />
==See Also==<br />
[[Quality Perception in Information Systems]]<br />
<br />
<br />
----<br />
<references /><br />
''(This Article was created by Christian Fürber on October 09th 2011)''</div>Admin