SemWebQuality.org - User contributions [en]

Semantic Web Quality Wiki:About

2015-08-27T17:37:24Z

Cfuerber:

SemWebQuality.org is a project dedicated to investigate the usefulness of Semantic Web technologies for data quality management.

This server is being operated by

Dr. Christian Fürber 

[https://iqinstitute.de Information Quality Institute GmbH] 
 
Blumenstr. 3 
D-85540 Haar, Germany 

'''Phone:''' +49 89 / 2155 4432-0 

'''Fax:''' +49 89 / 2155 4432-9 

'''E-mail:''' c.fuerber(at)iqinstitute.de

'''Website:''' [https://www.iqinstitute.de/ www.iqinstitute.de]

'''Blog:''' [https://www.iqinstitute.de/blog Information Quality Insider]

'''Imprint:''' [https://iqinstitute.de/impressum/ Impressum]

Semantic Web Quality Wiki:About

2015-08-27T17:36:50Z

Cfuerber:

SemWebQuality.org is a project dedicated to investigate the usefulness of Semantic Web technologies for data quality management.

This server is being operated by

Dr. Christian Fürber 

[https://iqinstitute.de Information Quality Institute GmbH] 
 
Blumenstr. 3 
D-85540 Haar, Germany 

'''Phone:''' +49 89 / 2155 4432-0 

'''Fax:''' +49 89 / 2155 4432-0 

'''E-mail:''' c.fuerber(at)iqinstitute.de

'''Website:''' https://www.iqinstitute.de/

'''Blog:''' [https://www.iqinstitute.de/blog Information Quality Insider]

'''Imprint:''' [https://iqinstitute.de/impressum/ Impressum]

Semantic Web Quality Wiki:About

2015-08-27T17:36:29Z

Cfuerber:

SemWebQuality.org is a project dedicated to investigate the usefulness of Semantic Web technologies for data quality management.

This server is being operated by

Dr. Christian Fürber 

[https://iqinstitute.de Information Quality Institute GmbH] 
 
Blumenstr. 3 
D-85540 Haar, Germany 

'''Phone:''' +49 89 / 2155 4432-0 

'''Fax:''' +49 89 / 2155 4432-0 

'''E-mail:''' c.fuerber(at)iqinstitute.de

'''Website:''' [https://www.iqinstitute.de/]

'''Blog:''' [https://www.iqinstitute.de/blog Information Quality Insider]

'''Imprint:''' [https://iqinstitute.de/impressum/ Impressum]

Semantic Web Quality Wiki:About

2015-08-27T17:35:42Z

Cfuerber:

SemWebQuality.org is a project dedicated to investigate the usefulness of Semantic Web technologies for data quality management.

This server is being operated by

Dr. Christian Fürber 

[https://iqinstitute.de Information Quality Institute GmbH] 
 
Blumenstr. 3 
D-85540 Haar, Germany 

'''Phone:''' +49 89 / 2155 4432-0 

'''Fax:''' +49 89 / 2155 4432-0 

'''E-mail:''' c.fuerber(at)iqinstitute.de

'''Website:''' [https://www.iqinstitute.de/blog Information Quality Insider]

'''Imprint:''' [https://iqinstitute.de/impressum/ Impressum]

Data Requirements Consistency Checks

2015-02-27T12:37:43Z

Cfuerber:

==Contradictory Unique Value Rules==

'''With generic superproperty'''
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>

SELECT
(?genprop AS ?GenericProperty)
(?dr1 AS ?UniqueRequirement)
(?dr2 AS ?NonUniqueRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 rdfs:subClassOf ?genclass .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 rdfs:subPropertyOf ?genprop .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 rdfs:subClassOf ?genclass .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 rdfs:subPropertyOf ?genprop .
?prop2 dqm:hasURI ?prop2URI .
FILTER(?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?dr2))
}
</syntaxhighlight>
'''Without generic superproperty'''
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?dr1 AS ?UniqueValueRequirement) (?dr2 AS ?InconsistentRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?tclassURI1 AS ?Class) (?tpropURI1 AS ?Property) (?dr1 AS ?DataRequirement)
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
}
ORDER BY ?tclassURI1 ?tpropURI1 ?dr1
</syntaxhighlight>

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
</syntaxhighlight>

Data Requirements Consistency Checks

2015-02-27T12:36:35Z

Cfuerber:

==Contradictory Unique Value Rules==

'''With generic superproperty'''
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>

SELECT (?genprop AS ?GenericProperty) (?dr1 AS ?UniqueRequirement) (?dr2 AS ?NonUniqueRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 rdfs:subClassOf ?genclass .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 rdfs:subPropertyOf ?genprop .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 rdfs:subClassOf ?genclass .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 rdfs:subPropertyOf ?genprop .
?prop2 dqm:hasURI ?prop2URI .
FILTER(?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?dr2))
}
</syntaxhighlight>
'''Without generic superproperty'''
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?dr1 AS ?UniqueValueRequirement) (?dr2 AS ?InconsistentRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?tclassURI1 AS ?Class) (?tpropURI1 AS ?Property) (?dr1 AS ?DataRequirement)
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
}
ORDER BY ?tclassURI1 ?tpropURI1 ?dr1
</syntaxhighlight>

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
</syntaxhighlight>

Data Requirements Consistency Checks

2015-02-27T12:35:19Z

Cfuerber:

==Contradictory Unique Value Rules==

'''Without generic superproperty'''
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>

SELECT (?dr1 AS ?UniqueRequirement) (?dr2 AS ?NonUniqueRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 rdfs:subClassOf ?genclass .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 rdfs:subPropertyOf ?genprop .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 rdfs:subClassOf ?genclass .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 rdfs:subPropertyOf ?genprop .
?prop2 dqm:hasURI ?prop2URI .
FILTER(?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?dr2))
}
</syntaxhighlight>
'''Without generic superproperty'''
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?dr1 AS ?UniqueValueRequirement) (?dr2 AS ?InconsistentRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?tclassURI1 AS ?Class) (?tpropURI1 AS ?Property) (?dr1 AS ?DataRequirement)
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
}
ORDER BY ?tclassURI1 ?tpropURI1 ?dr1
</syntaxhighlight>

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
</syntaxhighlight>

Data Requirements Consistency Checks

2015-02-27T12:34:59Z

Cfuerber:

==Contradictory Unique Value Rules==

'''Without generic superproperty'''
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
prefix rdfs:<http://www.w3.org/2000/01/rdf-schema#>

SELECT (?dr1 AS ?UniqueRequirement) (?dr2 AS ?NonUniqueRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 rdfs:subClassOf ?genclass .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 rdfs:subPropertyOf ?genprop .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 rdfs:subClassOf ?genclass .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 rdfs:subPropertyOf ?genprop .
?prop2 dqm:hasURI ?prop2URI .
FILTER(?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?dr2))
}
</syntaxhighlight>
'''Without generic superproperty'''
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?dr1 AS ?UniqueValueRequirement) (?dr2 AS ?InconsistentRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?tclassURI1 AS ?Class) (?tpropURI1 AS ?Property) (?dr1 AS ?DataRequirement)
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
}
ORDER BY ?tclassURI1 ?tpropURI1 ?dr1
</syntaxhighlight>

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
</syntaxhighlight>

Semantic Web Quality Wiki:About

2012-12-06T11:18:04Z

Cfuerber:

SemWebQuality.org is a project dedicated to investigate the usefulness of Semantic Web technologies for data quality management.

This server is being operated by

Dipl.-Kfm. Christian Fürber 

[http://www.unibw.de/ebusiness E-Business & Web Science Research Group] 
Universität der Bundeswehr München 
Werner-Heisenberg-Weg 39 
D-85579 Neubiberg, Germany 

'''Phone:''' +49-89-6004-4218 

'''Fax:''' +49-89-6004-4620 

'''E-mail:''' c.fuerber(at)unibw.de

'''Personal Blog:''' [http://www.iqinstitute-gmbh.de/blog Information Quality Insider]

Semantic Web Quality Wiki:About

2012-12-06T11:16:54Z

Cfuerber:

Semantic Web Quality Wiki:About

2012-12-06T11:16:24Z

Cfuerber:

Data Quality Assessment Reports

2012-07-01T16:12:03Z

Cfuerber: /* Completeness (OWL Full) */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="n3">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="n3">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Quality Monitoring Reports

2012-07-01T16:11:24Z

Cfuerber: /* Example 1.1: Missing Properties / Values (OWL Full Design) */

Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.

==Prerequisites==
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:

* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

==Data Quality Reports with SPARQL==

Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.

===Example 1.1: Missing Properties / Values (OWL Full Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||All instances with missing properties and values
|}
<syntaxhighlight lang="n3">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
{
?i a ?tclass .
FILTER NOT EXISTS{
?i ?tprop ?value .
}
}UNION{
?i a ?tclass .
?i ?tprop "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]

===Example 1.2: Missing Properties / Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||All instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?i a ?tclassURI .
NOT EXISTS{
?i ?tpropURI ?value .
}
}UNION{
?i a ?tclassURI .
?i ?tpropURI "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]

===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.
|-
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)
|-
|'''Output:'''||Instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?condvalue
WHERE {
?dqr a dqm:ConditionalPropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
{
?i a ?tclassURI .
?i ?cpropURI ?value
NOT EXISTS{
?i ?tpropURI ?value1 .
}
}UNION{
?i a ?tclassURI .
?i ?cpropURI ?value .
?i ?tpropURI "" .
}
FILTER (str(?value) = str(?condvalue)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 3: Syntax Violations (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||All instances with property values that have syntax violations
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:SyntaxRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}
</syntaxhighlight>
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.
|-
|'''Input:'''||All conditional syntax rules(in OWL DL Design).
|-
|'''Output:'''||All instances of the subset of the class that have syntax violations.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ConditionalSyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
?i a ?tclassURI .
?i ?cpropURI ?condvalueplain .
?i ?tpropURI ?value .
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 5: Out of Range Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have property values out of a specified range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||All instances with out of range values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 6: Illegal Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.
|-
|'''Input:'''||All legal value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with values not listed in the trusted reference.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 7: Uniqueness Violations (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances with property values that are not unique although they should.
|-
|'''Input:'''||All unique value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}
GROUP BY ?dqr ?i
</syntaxhighlight>

[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?dvalue ?value
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
?dqr dqm:hasCondition2 ?cond2 .
?cond2 dqm:conditionalProperty ?cprop2 .
?cprop2 dqm:hasURI ?cpropreal2 .
?cond2 dqm:equals ?condvalue2 .
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?cpropURI2 ?condvalue2 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 10: Outdated / Expired Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.
|-
|'''Output:'''||All instances with that are not valid anymore.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ExpiryRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}
</syntaxhighlight>

[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 11: Outdated / Not Updated Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.
|-
|'''Output:'''||All instances with that are potentially outdated.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}
</syntaxhighlight>

[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find duplicate instances.
|-
|'''Input:'''||Duplicate instance rules with 3 tested properties.
|-
|'''Output:'''||All instances that have identical values for the 3 tested properties.
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?i
WHERE {
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}
GROUP BY ?dqr
</syntaxhighlight>

===Example 13: Functional Dependency Violations (2 Reference Properties, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find functional dependency violations between two property values.
|-
|'''Input:'''||Functional dependency reference rules with 2 tested properties.
|-
|'''Output:'''||All instances that have violating dependencies between the two tested property values.
|}

<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepReferenceRule .
?dqr dqm:assessment "true"^^xsd:boolean .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:referenceClass ?trefclass .
?trefclass dqm:hasURI ?trefclassreal .
BIND (IRI(str(?trefclassreal)) AS ?trefclassURI) .
?dqr dqm:referenceProperty1 ?trefprop1 .
?trefprop1 dqm:hasURI ?trefpropreal1 .
BIND (IRI(str(?trefpropreal1)) AS ?trefpropURI1) .
?dqr dqm:referenceProperty2 ?trefprop2 .
?trefprop2 dqm:hasURI ?trefpropreal2 .
BIND (IRI(str(?trefpropreal2)) AS ?trefpropURI2) .
?i a ?tclassURI .
?i ?tpropURI1 ?value1 .
?i ?tpropURI2 ?value2 .
BIND(STRDT(str(?value1),xsd:string) AS ?value11 )
BIND(STRDT(str(?value2),xsd:string) AS ?value12 )
NOT EXISTS{
?i2 a ?trefclassURI .
?i2 ?trefpropURI1 ?value11 .
?i2 ?trefpropURI2 ?value12 .
}
}
</syntaxhighlight>

Data Quality Assessment Reports

2012-07-01T16:09:14Z

Cfuerber: /* Completeness (OWL DL) */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="n3">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Requirements Consistency Checks

2012-06-18T16:58:18Z

Cfuerber: /* Contradictory Unique Value Rules */

==Contradictory Unique Value Rules==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?dr1 AS ?UniqueValueRequirement) (?dr2 AS ?InconsistentRequirement)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?dr1 != ?dr2)
MINUS{
?dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?tclassURI1 AS ?Class) (?tpropURI1 AS ?Property) (?dr1 AS ?DataRequirement)
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
}
ORDER BY ?tclassURI1 ?tpropURI1 ?dr1
</syntaxhighlight>

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
<syntaxhighlight>

Data Requirements Consistency Checks

2012-06-18T16:56:21Z

Cfuerber: /* Contradictory Unique Value Rules */

==Contradictory Unique Value Rules==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?s AS ?uniquevaluereq) (?s2 AS ?nonuniquenessreq)
WHERE{
?dr1 a dqm:UniqueValueRule .
?dr1 dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?dr1 dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?dr2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && dr1 != dr2)
MINUS{
dr2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?tclassURI1 AS ?Class) (?tpropURI1 AS ?Property) (?dr1 AS ?DataRequirement)
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
}
ORDER BY ?tclassURI1 ?tpropURI1 ?dr1
</syntaxhighlight>

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
<syntaxhighlight>

Data Requirements Consistency Checks

2012-06-18T16:23:50Z

Cfuerber: /* Duplicate Property Requirements */

==Contradictory Unique Value Rules==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?s AS ?uniquevaluereq) (?s2 AS ?nonuniquenessreq)
WHERE{
?s a dqm:UniqueValueRule .
?s dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?s dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?s2 a dqm:PropertyRequirement .
?s2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?s2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?s != ?s2)
MINUS{
?s2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?tclassURI1 AS ?Class) (?tpropURI1 AS ?Property) (?dr1 AS ?DataRequirement)
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
}
ORDER BY ?tclassURI1 ?tpropURI1 ?dr1
</syntaxhighlight>

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
<syntaxhighlight>

Data Requirements Consistency Checks

2012-06-18T16:23:15Z

Cfuerber:

==Contradictory Unique Value Rules==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?s AS ?uniquevaluereq) (?s2 AS ?nonuniquenessreq)
WHERE{
?s a dqm:UniqueValueRule .
?s dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?s dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?s2 a dqm:PropertyRequirement .
?s2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?s2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?s != ?s2)
MINUS{
?s2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT (?tclassURI1 AS ?Class) (?tpropURI1 AS ?Property) (?dr1 AS ?DataRequirement)
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
}
ORDER BY ?tclassURI1 ?tpropURI1 ?dr1
<syntaxhighlight>

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
<syntaxhighlight>

Data Requirements Consistency Checks

2012-06-18T16:10:10Z

Cfuerber:

==Contradictory Unique Value Rules==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?s AS ?uniquevaluereq) (?s2 AS ?nonuniquenessreq)
WHERE{
?s a dqm:UniqueValueRule .
?s dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?s dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?s2 a dqm:PropertyRequirement .
?s2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?s2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?s != ?s2)
MINUS{
?s2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

==Duplicate Property Requirements==
<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dr1 ?dr2
WHERE {
?dr1 a dqm:PropertyRequirement .
?dr1 dqm:testedClass ?tclass1 .
?tclass1 dqm:hasURI ?tclassURI1 .
?dr1 dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropURI1 .
OPTIONAL{
?dr2 a dqm:PropertyRequirement .
?dr2 dqm:testedClass ?tclass2 .
?tclass2 dqm:hasURI ?tclassURI2 .
?dr2 dqm:testedProperty1 ?tprop2 .
?tprop2 dqm:hasURI ?tpropURI2 .
FILTER(?dr1!=?dr2 && str(?tclassURI1) = str(?tclassURI2) && str(?tpropURI1) = str(?tpropURI2))
}
}
<syntaxhighlight>

Data Requirements Consistency Checks

2012-06-17T10:43:15Z

Cfuerber:

Data Requirements Consistency Checks

2012-06-17T10:34:17Z

Cfuerber:

<syntaxhighlight lang="n3">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?s AS ?uniquevaluereq) (?s2 AS ?nonuniquenessreq)
WHERE{
?s a dqm:UniqueValueRule .
?s dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?s dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?s2 a dqm:PropertyRequirement .
?s2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?s2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?s != ?s2)
MINUS{
?s2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

Data Requirements Consistency Checks

2012-06-17T10:33:36Z

Cfuerber:

Data Requirements Consistency Checks

2012-06-17T10:31:47Z

Cfuerber: Created page with "<syntaxhighlight lang="sparql"> PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#> SELECT (?s AS ?uniquevaluereq) (?s2 AS ?nonuniquenessreq) WHERE{ ?s a dqm:UniqueValueRule . ..."

<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
SELECT (?s AS ?uniquevaluereq) (?s2 AS ?nonuniquenessreq)
WHERE{
?s a dqm:UniqueValueRule .
?s dqm:testedClass ?class1 .
?class1 dqm:hasURI ?class1URI .
?s dqm:testedProperty1 ?prop1 .
?prop1 dqm:hasURI ?prop1URI .
OPTIONAL{
?s2 a dqm:PropertyRequirement .
?s2 dqm:testedClass ?class2 .
?class2 dqm:hasURI ?class2URI .
?s2 dqm:testedProperty1 ?prop2 .
?prop2 dqm:hasURI ?prop2URI .
FILTER(str(?prop1URI) = str(?prop2URI) && str(?class1URI) = str(?class2URI) && ?s != ?s2)
MINUS{
?s2 a dqm:UniqueValueRule
}
}
FILTER(bound(?prop2URI))
}
</syntaxhighlight>

SemWebQuality.org

2012-06-04T15:45:49Z

Cfuerber:

__NOTOC__

{{DISPLAYTITLE:{{FULLPAGENAME}}}}
'''Welcome to SemWebQuality.org'''
This website provides information and tools about data quality in Semantic Web architectures.
<Table cellpadding="3">
<tr>
<td Width="50%">
{{BigHeading
|Title=Data Quality Constraint Library for [http://spinrdf.org SPIN] }}
The data quality constraints library may be used (1) for the identification of data quality problems and (2) for quality assurance during data entry.
* [http://semwebquality.org/documentation/primer/ DQ-Constraints Library Primer] 
* [http://semwebquality.org/ontologies/dq-constraints.html DQ-Constraints Library Documentation] 
* [http://semwebquality.org/ontologies/dq-constraints.rdf Download DQ-Constraints in RDF] 
* [http://groups.google.com/group/dq-constraints-library DQ-Constraints Mailing List]
</td>
<td Width="50%">
{{BigHeading
|Title=Data Quality Management Vocabulary }}
An Ontology for Data Requirements Management, Data Quality Monitoring, Data Quality Assessment, and Data Cleansing. 
* [[DQM-Vocabulary Primer]]
* [http://purl.org/dqm-vocabulary/v1/dqm Latest Technical Specification of DQM-Vocabulary]
* [http://semwebquality.org/dqm-vocabulary/v1/dqm.owl Latest DQM-Vocabulary in OWL]
* [http://groups.google.com/group/dqm-vocabulary DQM-Vocabulary Mailing List]
</td>
</tr>
<tr>
<td>
{{BigHeading
|Title=Information around Data Quality }}
* What is [[Data Quality]]?
* [[Data Requirements]]
* [[Data Quality Management]]
* [[Quality Perception in Information Systems]]
* [http://www.dqglossary.com/ Data Quality Glossary]
</td>
<td>
{{BigHeading
|Title= Semantic Web Quality Links}}
* [http://pedantic-web.org/ Pedantic Web Group] 
* [http://qualitywebdata.org/ Qualitywebdata.org] 
* [http://sourceforge.net/apps/mediawiki/trdf/index.php?title=Quality_Criteria_for_Linked_Data_sources Quality Criteria for Linked Data Sources (Findings by Annika Flemming & Olaf Hartig)]
* [http://patterns.dataincubator.org/book/index.html Linked Data Patterns Book]
</td>
</tr>
<tr>
<td>
{{BigHeading
|Title=Ontology Quality }}
http://ontologydesignpatterns.org/ 
http://www.ontoclean.org/
</td>
<td>
{{BigHeading
|Title=Data Quality Discussions }}
http://answers.semanticweb.com/questions/1072/quality-indicators-for-linked-data-datasets?sort=votes
</td>
</tr>
</Table>

Data Quality Monitoring Reports

2012-05-16T19:41:04Z

Cfuerber: /* Example 1.1: Missing Properties / Values (OWL Full Design) */

Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.

==Prerequisites==
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:

* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

==Data Quality Reports with SPARQL==

Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.

===Example 1.1: Missing Properties / Values (OWL Full Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||All instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
{
?i a ?tclass .
FILTER NOT EXISTS{
?i ?tprop ?value .
}
}UNION{
?i a ?tclass .
?i ?tprop "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]

===Example 1.2: Missing Properties / Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||All instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?i a ?tclassURI .
NOT EXISTS{
?i ?tpropURI ?value .
}
}UNION{
?i a ?tclassURI .
?i ?tpropURI "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]

===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.
|-
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)
|-
|'''Output:'''||Instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?condvalue
WHERE {
?dqr a dqm:ConditionalPropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
{
?i a ?tclassURI .
?i ?cpropURI ?value
NOT EXISTS{
?i ?tpropURI ?value1 .
}
}UNION{
?i a ?tclassURI .
?i ?cpropURI ?value .
?i ?tpropURI "" .
}
FILTER (str(?value) = str(?condvalue)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 3: Syntax Violations (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||All instances with property values that have syntax violations
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:SyntaxRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}
</syntaxhighlight>
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.
|-
|'''Input:'''||All conditional syntax rules(in OWL DL Design).
|-
|'''Output:'''||All instances of the subset of the class that have syntax violations.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ConditionalSyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
?i a ?tclassURI .
?i ?cpropURI ?condvalueplain .
?i ?tpropURI ?value .
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 5: Out of Range Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have property values out of a specified range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||All instances with out of range values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 6: Illegal Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.
|-
|'''Input:'''||All legal value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with values not listed in the trusted reference.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 7: Uniqueness Violations (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances with property values that are not unique although they should.
|-
|'''Input:'''||All unique value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}
GROUP BY ?dqr ?i
</syntaxhighlight>

[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?dvalue ?value
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
?dqr dqm:hasCondition2 ?cond2 .
?cond2 dqm:conditionalProperty ?cprop2 .
?cprop2 dqm:hasURI ?cpropreal2 .
?cond2 dqm:equals ?condvalue2 .
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?cpropURI2 ?condvalue2 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 10: Outdated / Expired Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.
|-
|'''Output:'''||All instances with that are not valid anymore.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ExpiryRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}
</syntaxhighlight>

[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 11: Outdated / Not Updated Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.
|-
|'''Output:'''||All instances with that are potentially outdated.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}
</syntaxhighlight>

[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find duplicate instances.
|-
|'''Input:'''||Duplicate instance rules with 3 tested properties.
|-
|'''Output:'''||All instances that have identical values for the 3 tested properties.
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?i
WHERE {
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}
GROUP BY ?dqr
</syntaxhighlight>

===Example 13: Functional Dependency Violations (2 Reference Properties, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find functional dependency violations between two property values.
|-
|'''Input:'''||Functional dependency reference rules with 2 tested properties.
|-
|'''Output:'''||All instances that have violating dependencies between the two tested property values.
|}

<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepReferenceRule .
?dqr dqm:assessment "true"^^xsd:boolean .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:referenceClass ?trefclass .
?trefclass dqm:hasURI ?trefclassreal .
BIND (IRI(str(?trefclassreal)) AS ?trefclassURI) .
?dqr dqm:referenceProperty1 ?trefprop1 .
?trefprop1 dqm:hasURI ?trefpropreal1 .
BIND (IRI(str(?trefpropreal1)) AS ?trefpropURI1) .
?dqr dqm:referenceProperty2 ?trefprop2 .
?trefprop2 dqm:hasURI ?trefpropreal2 .
BIND (IRI(str(?trefpropreal2)) AS ?trefpropURI2) .
?i a ?tclassURI .
?i ?tpropURI1 ?value1 .
?i ?tpropURI2 ?value2 .
BIND(STRDT(str(?value1),xsd:string) AS ?value11 )
BIND(STRDT(str(?value2),xsd:string) AS ?value12 )
NOT EXISTS{
?i2 a ?trefclassURI .
?i2 ?trefpropURI1 ?value11 .
?i2 ?trefpropURI2 ?value12 .
}
}
</syntaxhighlight>

Data Quality Assessment Reports

2012-05-16T19:39:33Z

Cfuerber: /* Completeness (OWL DL) */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Quality Assessment Reports

2012-05-16T19:39:15Z

Cfuerber: /* Completeness (OWL Full) */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="n3">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Quality Assessment Reports

2012-05-16T19:37:52Z

Cfuerber: /* Completeness (OWL Full) */

Data Quality Assessment Reports

2012-05-16T19:37:37Z

Cfuerber: /* Completeness (OWL DL) */

Data Quality Assessment Reports

2012-05-16T19:37:10Z

Cfuerber: /* Completeness (OWL Full) */

Data Quality Assessment Reports

2012-05-16T19:36:48Z

Cfuerber: /* Completeness (OWL DL) */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="n3">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Quality Assessment Reports

2012-05-16T19:36:11Z

Cfuerber: /* Completeness (OWL Full) */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Quality Assessment Reports

2012-05-16T19:35:48Z

Cfuerber: /* Completeness (OWL DL) */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
FILTER NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Quality Assessment Reports

2012-05-16T19:35:30Z

Cfuerber: /* Completeness (OWL DL) */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Quality Assessment Reports

2012-05-16T19:35:10Z

Cfuerber: /* Completeness */

Data Quality Scores can be used to get a quick impression about the quality state of a data source. Below you can find queries that can calculate data quality scores based on pre-defined data requirements on the fly.

=Prerequisites=
With the DQM-Vocabulary you can easily create Data Quality Scores. All you need is a SPARQL-Endpoint that supports SPARQL 1.1 or SPARQL extensions that cover datatype conversion and conversion of strings to URIs. Before you can automatically analyze your data for requirement violations, you need to perform the following steps:
* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

=Data Quality Score Calculations=

==Completeness (OWL DL)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Completeness (OWL Full)==
{|class="wikitable"
|'''Scenario:'''||Calculate the [[Completeness]] of a property based on the ratio of all instances of a class and instances that miss a specific property or property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||[[Completeness]] Scores per Property Completeness Requirement
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?s) AS ?violations) (COUNT(?s2) AS ?total)
(((?total - ?violations)/?total) AS ?completeness)
WHERE {
{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?s a ?tclassURI .
NOT EXISTS{
?s ?tpropURI ?value .
}
}UNION{
?s a ?tclassURI .
?s ?tpropURI "" .
}
}UNION{
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassURI .
?tprop dqm:hasURI ?tpropURI .
?s2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Depth==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of a property based on the ratio of all instances of a class and instances that have non-unique property values.
|-
|'''Input:'''||All unique value rules (in OWL DL Design)
|-
|'''Output:'''||Uniqueness Scores per unique value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI ((COUNT(?i)/2) AS ?violations) (COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessDepth)
WHERE {
{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}UNION{
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i3 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Uniqueness in Scope==
{|class="wikitable"
|'''Scenario:'''||Calculate the uniqueness of class instances based on the ratio of all instances of a class and duplicate instances.
|-
|'''Input:'''||All duplicate instances rules (Example case: 3 tested Properties, OWL DL Design)
|-
|'''Output:'''||Uniqueness in scope scores for classes
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?tclassURI
((COUNT(?i)/2) AS ?violations)
(COUNT(?i3) AS ?total)
(((?total - ?violations)/?total) AS ?uniquenessScope)
WHERE {
{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}UNION{
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i3 a ?tclassURI .
}
}
GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Syntactic Accuracy==

===Based on Syntax Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values with syntax rule violations.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per syntax rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}UNION{
?dqr a dqm:SyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are not listed as legal values.
|-
|'''Input:'''||All legal value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}UNION{
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

===Based on Legal Value Range Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Syntactic Accuracy]] of a property based on the ratio of all instances of a class and instances that have property values that are out of a legal range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Syntactic Accuracy]] per legal value range rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI ?tpropURI (COUNT(?i) AS ?violations) (COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?syntacticAccuracy)
WHERE {
{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}UNION{
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI ?tpropURI
</syntaxhighlight>

==Semantic Accuracy==

===Based on Functional Dependent Value Rule (1 Condition)===
{|class="wikitable"
|'''Scenario:'''||Calculate an approximation of [[Semantic Accuracy]] of an instance regarding the value of a certain property based on the ratio of all instances of a class and instances that have property value combinations that are incorrect.
|-
|'''Input:'''||All functional dependent value rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Semantic Accuracy]] per functional dependent value rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?semanticAccuracy)
WHERE {
{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}UNION{
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i2 a ?tclassURI .
?i2 ?cpropURI1 ?condvalue1 .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

==Timeliness==
===Based on Expiry Rules===
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and expired instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design)
|-
|'''Output:'''||Score for [[Timeliness]] per expiry rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}UNION{
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

===Based on Update Rules===
'''Please Note:''' This rule requires the SPARQL extension '''dqf:requiredTimestamp''' in order to work.
{|class="wikitable"
|'''Scenario:'''||Calculate [[Timeliness]] of class instances based on the ratio of all instances of a class and instances that have not been updated in a specific period of time.
|-
|'''Input:'''||All update rules (in OWL DL Design) and the timestamps of the tested instances
|-
|'''Output:'''||Score for [[Timeliness]] per update rule
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?tclassURI
(COUNT(?i) AS ?violations)
(COUNT(?i2) AS ?total)
(((?total - ?violations)/?total) AS ?timeliness)
WHERE {
{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}UNION{
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i2 a ?tclassURI .
}
}GROUP BY ?dqr ?tclassURI
</syntaxhighlight>

Data Quality Monitoring Reports

2012-03-28T14:58:04Z

Cfuerber: /* Example 1.1: Missing Properties / Values (OWL Full Design) */

Create Data Requirements

2012-03-28T12:28:00Z

Cfuerber: /* Examples of Data Requirements */

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Sample Dataset used in Examples ==
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

==Syntax Of Examples==

The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].

== Examples of Data Requirements ==

===Example 1: Property Completeness Rule===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<syntaxhighlight lang="n3">
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</syntaxhighlight>
[[Data Quality Monitoring Reports#Example 1.1: Missing Properties / Values (OWL Full Design)|Click here to learn how to generate a monitoring report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<syntaxhighlight lang="n3">
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</syntaxhighlight>
[[Data Quality Monitoring Reports#Example 1.2: Missing Properties / Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<syntaxhighlight lang="n3">
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</syntaxhighlight>

'''2. Define the Conditional Property Completeness Rule'''

<syntaxhighlight lang="n3">
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</syntaxhighlight>
[[Data Quality Monitoring Reports#Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<syntaxhighlight lang="n3">
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</syntaxhighlight>

[[Data Quality Monitoring Reports#Example 3: Syntax Violations (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<syntaxhighlight lang="n3">
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</syntaxhighlight>

'''2.Define Conditional Syntax Rule'''
<syntaxhighlight lang="n3">
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</syntaxhighlight>

[[Data Quality Monitoring Reports#Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<syntaxhighlight lang="n3">
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</syntaxhighlight>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

[[Data Quality Monitoring Reports#Example 5: Out of Range Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<syntaxhighlight lang="n3">
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</syntaxhighlight>

[[Data Quality Monitoring Reports#Example 6: Illegal Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<syntaxhighlight lang="n3">
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</syntaxhighlight>

[[Data Quality Monitoring Reports#Example 7: Uniqueness Violations (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<syntaxhighlight lang="n3">
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</syntaxhighlight>

[[Data Quality Monitoring Reports#Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<syntaxhighlight lang="n3">
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</syntaxhighlight>

[[Data Quality Monitoring Reports#Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<syntaxhighlight lang="n3">
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</syntaxhighlight>

[[Data Quality Monitoring Reports#Example 10: Outdated / Expired Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 11: Update Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<syntaxhighlight lang="n3">
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</syntaxhighlight>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.

[[Data Quality Monitoring Reports#Example 11: Outdated / Not Updated Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]

Create Data Requirements

2012-03-28T12:26:23Z

Cfuerber: /* Example 1: Property Completeness Rule */

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Sample Dataset used in Examples ==
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

==Syntax Of Examples==

The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].

== Examples of Data Requirements ==

===Example 1: Property Completeness Rule===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<syntaxhighlight lang="n3">
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</syntaxhighlight>
[[Data Quality Monitoring Reports#Example 1.1: Missing Properties / Values (OWL Full Design)|Click here to learn how to generate a monitoring report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Data Quality Monitoring Reports#Example 1.2: Missing Properties / Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2. Define the Conditional Property Completeness Rule'''

<pre>
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>
[[Data Quality Monitoring Reports#Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<pre>
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</pre>

[[Data Quality Monitoring Reports#Example 3: Syntax Violations (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2.Define Conditional Syntax Rule'''
<pre>
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

[[Data Quality Monitoring Reports#Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<pre>
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</pre>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

[[Data Quality Monitoring Reports#Example 5: Out of Range Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<pre>
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

[[Data Quality Monitoring Reports#Example 6: Illegal Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<pre>
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</pre>

[[Data Quality Monitoring Reports#Example 7: Uniqueness Violations (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

[[Data Quality Monitoring Reports#Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

[[Data Quality Monitoring Reports#Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<pre>
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</pre>

[[Data Quality Monitoring Reports#Example 10: Outdated / Expired Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]] 

===Example 11: Update Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<pre>
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</pre>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.

[[Data Quality Monitoring Reports#Example 11: Outdated / Not Updated Values (OWL DL Design)|Click here to learn how to generate a monitoring report from this]]

Data Quality Monitoring Reports

2012-03-28T12:25:31Z

Cfuerber: /* Example 1.1: Missing Properties / Values (OWL Full Design) */

Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.

==Prerequisites==
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:

* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

==Data Quality Reports with SPARQL==

Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.

===Example 1.1: Missing Properties / Values (OWL Full Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||All instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
{
?i a ?tclass .
NOT EXISTS{
?i ?tprop ?value .
}
}UNION{
?i a ?tclass .
?i ?tprop "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]

===Example 1.2: Missing Properties / Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||All instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?i a ?tclassURI .
NOT EXISTS{
?i ?tpropURI ?value .
}
}UNION{
?i a ?tclassURI .
?i ?tpropURI "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]

===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.
|-
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)
|-
|'''Output:'''||Instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?condvalue
WHERE {
?dqr a dqm:ConditionalPropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
{
?i a ?tclassURI .
?i ?cpropURI ?value
NOT EXISTS{
?i ?tpropURI ?value1 .
}
}UNION{
?i a ?tclassURI .
?i ?cpropURI ?value .
?i ?tpropURI "" .
}
FILTER (str(?value) = str(?condvalue)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 3: Syntax Violations (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||All instances with property values that have syntax violations
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:SyntaxRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}
</syntaxhighlight>
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.
|-
|'''Input:'''||All conditional syntax rules(in OWL DL Design).
|-
|'''Output:'''||All instances of the subset of the class that have syntax violations.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ConditionalSyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
?i a ?tclassURI .
?i ?cpropURI ?condvalueplain .
?i ?tpropURI ?value .
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 5: Out of Range Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have property values out of a specified range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||All instances with out of range values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 6: Illegal Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.
|-
|'''Input:'''||All legal value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with values not listed in the trusted reference.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 7: Uniqueness Violations (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances with property values that are not unique although they should.
|-
|'''Input:'''||All unique value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}
GROUP BY ?dqr ?i
</syntaxhighlight>

[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?dvalue ?value
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
?dqr dqm:hasCondition2 ?cond2 .
?cond2 dqm:conditionalProperty ?cprop2 .
?cprop2 dqm:hasURI ?cpropreal2 .
?cond2 dqm:equals ?condvalue2 .
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?cpropURI2 ?condvalue2 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 10: Outdated / Expired Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.
|-
|'''Output:'''||All instances with that are not valid anymore.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ExpiryRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}
</syntaxhighlight>

[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 11: Outdated / Not Updated Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.
|-
|'''Output:'''||All instances with that are potentially outdated.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}
</syntaxhighlight>

[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find duplicate instances.
|-
|'''Input:'''||Duplicate instance rules with 3 tested properties.
|-
|'''Output:'''||All instances that have identical values for the 3 tested properties.
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?i
WHERE {
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}
GROUP BY ?dqr
</syntaxhighlight>

===Example 13: Functional Dependency Violations (2 Reference Properties, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find functional dependency violations between two property values.
|-
|'''Input:'''||Functional dependency reference rules with 2 tested properties.
|-
|'''Output:'''||All instances that have violating dependencies between the two tested property values.
|}

<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepReferenceRule .
?dqr dqm:assessment "true"^^xsd:boolean .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:referenceClass ?trefclass .
?trefclass dqm:hasURI ?trefclassreal .
BIND (IRI(str(?trefclassreal)) AS ?trefclassURI) .
?dqr dqm:referenceProperty1 ?trefprop1 .
?trefprop1 dqm:hasURI ?trefpropreal1 .
BIND (IRI(str(?trefpropreal1)) AS ?trefpropURI1) .
?dqr dqm:referenceProperty2 ?trefprop2 .
?trefprop2 dqm:hasURI ?trefpropreal2 .
BIND (IRI(str(?trefpropreal2)) AS ?trefpropURI2) .
?i a ?tclassURI .
?i ?tpropURI1 ?value1 .
?i ?tpropURI2 ?value2 .
BIND(STRDT(str(?value1),xsd:string) AS ?value11 )
BIND(STRDT(str(?value2),xsd:string) AS ?value12 )
NOT EXISTS{
?i2 a ?trefclassURI .
?i2 ?trefpropURI1 ?value11 .
?i2 ?trefpropURI2 ?value12 .
}
}
</syntaxhighlight>

Data Quality Monitoring Reports

2012-03-28T12:25:00Z

Cfuerber: /* Example 1.1: Missing Properties / Values (OWL Full Design) */

Data Quality Monitoring Reports are an excellent instrument to quickly identify instances with requirement violations.

==Prerequisites==
With the DQM-Vocabulary you can easily create Data Quality Monitoring reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:

* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

==Data Quality Reports with SPARQL==

Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.

===Example 1.1: Missing Properties / Values (OWL Full Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||All instances with missing properties and values
|}
<syntaxhighlight lang="n3">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
{
?i a ?tclass .
NOT EXISTS{
?i ?tprop ?value .
}
}UNION{
?i a ?tclass .
?i ?tprop "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]

===Example 1.2: Missing Properties / Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||All instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?i a ?tclassURI .
NOT EXISTS{
?i ?tpropURI ?value .
}
}UNION{
?i a ?tclassURI .
?i ?tpropURI "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]

===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.
|-
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)
|-
|'''Output:'''||Instances with missing properties and values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?condvalue
WHERE {
?dqr a dqm:ConditionalPropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
{
?i a ?tclassURI .
?i ?cpropURI ?value
NOT EXISTS{
?i ?tpropURI ?value1 .
}
}UNION{
?i a ?tclassURI .
?i ?cpropURI ?value .
?i ?tpropURI "" .
}
FILTER (str(?value) = str(?condvalue)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 3: Syntax Violations (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||All instances with property values that have syntax violations
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:SyntaxRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}
</syntaxhighlight>
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.
|-
|'''Input:'''||All conditional syntax rules(in OWL DL Design).
|-
|'''Output:'''||All instances of the subset of the class that have syntax violations.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ConditionalSyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
?i a ?tclassURI .
?i ?cpropURI ?condvalueplain .
?i ?tpropURI ?value .
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 5: Out of Range Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have property values out of a specified range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||All instances with out of range values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 6: Illegal Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.
|-
|'''Input:'''||All legal value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with values not listed in the trusted reference.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 7: Uniqueness Violations (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances with property values that are not unique although they should.
|-
|'''Input:'''||All unique value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}
GROUP BY ?dqr ?i
</syntaxhighlight>

[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?dvalue ?value
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
?dqr dqm:hasCondition2 ?cond2 .
?cond2 dqm:conditionalProperty ?cprop2 .
?cprop2 dqm:hasURI ?cpropreal2 .
?cond2 dqm:equals ?condvalue2 .
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?cpropURI2 ?condvalue2 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 10: Outdated / Expired Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.
|-
|'''Output:'''||All instances with that are not valid anymore.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ExpiryRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}
</syntaxhighlight>

[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 11: Outdated / Not Updated Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.
|-
|'''Output:'''||All instances with that are potentially outdated.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(str(?duration))) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}
</syntaxhighlight>

[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 12: Duplicate Instance Rule (3 Properties, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find duplicate instances.
|-
|'''Input:'''||Duplicate instance rules with 3 tested properties.
|-
|'''Output:'''||All instances that have identical values for the 3 tested properties.
|}
<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?i
WHERE {
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:testedProperty3 ?tprop3 .
?tprop3 dqm:hasURI ?tpropreal3 .
BIND (IRI(str(?tpropreal3)) AS ?tpropURI3) .
?i a ?tclassURI .
?i ?tpropURI1 ?value11 .
?i2 ?tpropURI1 ?value12 .
?i ?tpropURI2 ?value21 .
?i2 ?tpropURI2 ?value22 .
?i ?tpropURI3 ?value31 .
?i2 ?tpropURI3 ?value32 .
FILTER(?i!=?i2 && str(?value11) = str(?value12) && str(?value21) = str(?value22) && str(?value31) = str(?value32))
}
GROUP BY ?dqr
</syntaxhighlight>

===Example 13: Functional Dependency Violations (2 Reference Properties, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find functional dependency violations between two property values.
|-
|'''Input:'''||Functional dependency reference rules with 2 tested properties.
|-
|'''Output:'''||All instances that have violating dependencies between the two tested property values.
|}

<syntaxhighlight lang="sparql">
PREFIX dqm:<http://purl.org/dqm-vocabulary/v1.1/dqm#>
PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepReferenceRule .
?dqr dqm:assessment "true"^^xsd:boolean .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:testedProperty1 ?tprop1 .
?tprop1 dqm:hasURI ?tpropreal1 .
BIND (IRI(str(?tpropreal1)) AS ?tpropURI1) .
?dqr dqm:testedProperty2 ?tprop2 .
?tprop2 dqm:hasURI ?tpropreal2 .
BIND (IRI(str(?tpropreal2)) AS ?tpropURI2) .
?dqr dqm:referenceClass ?trefclass .
?trefclass dqm:hasURI ?trefclassreal .
BIND (IRI(str(?trefclassreal)) AS ?trefclassURI) .
?dqr dqm:referenceProperty1 ?trefprop1 .
?trefprop1 dqm:hasURI ?trefpropreal1 .
BIND (IRI(str(?trefpropreal1)) AS ?trefpropURI1) .
?dqr dqm:referenceProperty2 ?trefprop2 .
?trefprop2 dqm:hasURI ?trefpropreal2 .
BIND (IRI(str(?trefpropreal2)) AS ?trefpropURI2) .
?i a ?tclassURI .
?i ?tpropURI1 ?value1 .
?i ?tpropURI2 ?value2 .
BIND(STRDT(str(?value1),xsd:string) AS ?value11 )
BIND(STRDT(str(?value2),xsd:string) AS ?value12 )
NOT EXISTS{
?i2 a ?trefclassURI .
?i2 ?trefpropURI1 ?value11 .
?i2 ?trefpropURI2 ?value12 .
}
}
</syntaxhighlight>

Quality Perception in Information Systems

2011-10-27T17:01:14Z

Cfuerber:

The quality of information is usually perceived by accessing multiple objects, namely
* Hardware (e.g. a computer),
* Application (e.g. query software, web browsers, search engines, etc.),
* Schema / Ontology (i.e. the structure in which the data is stored), and
* Data itself

[[Image:Perception_objects.png|300px|center]]

In order to find the root cause of information quality problems, it is important to be aware about the above layers. E.g. performace issues regarding data access may often be referred to insufficient configuration of the application or lack of hardware performance.

==See Also==
What is [[Data Quality]]

Data Quality Monitoring Reports (DEPRECATED)

2011-10-20T20:56:03Z

Cfuerber: moved Data Quality Monitoring Reports to Data Quality Monitoring Reports (DEPRECATED)

'''This site is currently under construction! In fact, many of these queries won't work since they depend on a different release of the DQM-Vocabulary'''

On this page you find [[Wikipedia:SPARQL|SPARQL]] queries that can be used to generate reports with instances that do not conform to the specified data requirements. Before you can use these queries you need to create data quality rules with the help of the [http://purl.org/dqm-vocabulary/v1/dqm DQM-Vocabulary]. The queries on this page only use elements of the dqm-voabulary so that they work for any datasets as long as the data requirements are specified with the [http://purl.org/dqm-vocabulary/v1/dqm DQM-Vocabulary].

__TOC__

== Completeness Reports ==
=== Conditional Value Rule ===
'''Addressed DQ-Problems:''' Missing Values
<syntaxhighlight lang="sparql">
SELECT ?i ?condvalue
WHERE {
?dqr a ?ConditionalValueRule .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
?i a ?tclassURI .
?dqr dqm:hasCondition ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
BIND (smf:buildURI(?cpropreal) AS ?cpropURI) .
?i ?cpropURI ?value .
FILTER (str(?value) = str(?condvalue)) .
?i ?tpropURI "" .
}
</syntaxhighlight>

=== Conditional Property Rule ===
'''Addressed DQ-Problems:''' Missing Conceptual Elements
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?condvalue
WHERE {
?dqr a dqm:ConditionalPropertyRule .
?dqr dqm:testedProperty ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
?i a ?tclassURI .
?dqr dqm:hasCondition ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
BIND (smf:buildURI(?cpropreal) AS ?cpropURI) .
?i ?cpropURI ?value .
FILTER (str(?value) = str(?condvalue)) .
NOT EXISTS {
?i ?tpropURI ?mandatoryvalue .
} .
}
</syntaxhighlight>

=== Missing Value Rule ===
'''Addressed DQ-Problems:''' Missing Values
<syntaxhighlight lang="sparql">
SELECT ?i
WHERE {
?dqr a dqm:MissingValueRule .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI "" .
}
</syntaxhighlight>

=== Missing Property Rule ===
'''Addressed DQ-Problems:''' Missing Conceptual Elements
<syntaxhighlight lang="sparql">
SELECT ?i
WHERE {
?dqr a dqm:MissingPropertyRule .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
?i a ?tclassURI .
NOT EXISTS{
?i ?tpropURI ?value .
}
}
</syntaxhighlight>

== Syntactical Accuracy Reports ==
=== Legal Value Rule ===
'''Addressed DQ-Problems:''' Invalid Characters, Character Alignment Violations, Embedded Values, Meaningless Values, (False Values, Outdated Values, Imprecise Values)
<syntaxhighlight lang="sparql">
SELECT ?i ?dqr
WHERE {
?dqr a dqm:LegalValueRule .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (smf:buildURI(?rpropvalue) AS ?rpropURI) .
BIND (smf:buildURI(?rclassvalue) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?s2 a ?rclassURI .
?s2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}
</syntaxhighlight>

=== Legal Value Range Rule ===

<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:testedClass ?tclass .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (smf:cast(?value, xsd:float) > ?upperLimit || smf:cast(?value, xsd:float) < ?lowerLimit) .
}
</syntaxhighlight>

=== Syntax Rule ===

<syntaxhighlight lang="sparql">
SELECT ?dqr ?regex ?i
WHERE {
?dqr a dqm:SyntaxRule .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
OPTIONAL{
?i a ?tclassURI .
?i ?tpropURI ?value .
}
FILTER (!regex(str(?value), ?regex)) .
}
</syntaxhighlight>

=== Conditional Syntax Rule ===

<syntaxhighlight lang="sparql">
SELECT ?dqr ?regex ?i
WHERE {
?dqr a dqm:ConditionalSyntaxRule .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
BIND (smf:buildURI(?cpropreal) AS ?cpropURI) .
?i a ?tclassURI .
?i ?cpropURI ?condvalueplain .
?i ?tpropURI ?value .
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .
}
</syntaxhighlight>

== Semantic Accuracy Reports ==
=== Functional Dependency Rule (2 AND Conditions) ===

<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?dvaluereal ?value
WHERE {
?dqr a dqm:FunctionalDependencyRule .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
?cond1 dqm:and ?cond2 .
?cond2 dqm:conditionalProperty ?cprop2 .
?cprop2 dqm:hasURI ?cpropreal2 .
?cond2 dqm:equals ?condvalue2 .
BIND (smf:buildURI(?tpropreal) AS ?tpropURI) .
BIND (smf:buildURI(?tclassreal) AS ?tclassURI) .
BIND (smf:buildURI(?cpropreal1) AS ?cpropURI1) .
BIND (smf:buildURI(?cpropreal2) AS ?cpropURI2) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?cpropURI2 ?condvalue2 .
?dqr dqm:hasDependentValue ?dvalue .
?dvalue dqm:equals ?dvaluereal .
?i ?tpropURI ?value .
FILTER (str(?dvaluereal) != str(?value)) .
}
</syntaxhighlight>

== Uniqueness Reports ==
=== Duplicate Instance Rule ===

<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:DuplicateInstanceRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
?i2 ?tpropURI ?value2 .
FILTER(?i!=?i2 && str(?value) = str(?value2))
}
GROUP BY ?dqr ?i
</syntaxhighlight>

== Timeliness Reports ==
=== Expiry Rule ===

<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?expiry ?today
WHERE {
?dqr a dqm:ExpiryRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
LET(?today:=afn:now()) .
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}
</syntaxhighlight>

=== Update Rule ===

<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?timestamp
WHERE {
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
LET(?expectedUpdate:=dqf:requiredTimestamp(?duration))
FILTER(?expectedUpdate>?timestamp)
}
</syntaxhighlight>

Data Quality Monitoring Reports

2011-10-20T20:50:53Z

Cfuerber:

'''This site is currently under construction!''' 
Problem reports are an excellent instrument to quickly identify instances with requirement violations.

==Prerequisites==
With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:

* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

==Data Requirement Violation Reports with SPARQL==

Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.

===Example 1.1: Missing Properties / Values (OWL Full Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||All instances with missing values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
{
?i a ?tclass .
NOT EXISTS{
?i ?tprop ?value .
}
}UNION{
?i a ?tclass .
?i ?tprop "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]

===Example 1.2: Missing Properties / Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||All instances with missing values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?i a ?tclassURI .
NOT EXISTS{
?i ?tpropURI ?value .
}
}UNION{
?i a ?tclassURI .
?i ?tpropURI "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]

===Example 2: Conditional Missing Properties / Values (1 Condition, OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.
|-
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)
|-
|'''Output:'''||Instances with missing values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?condvalue
WHERE {
?dqr a dqm:ConditionalPropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
{
?i a ?tclassURI .
?i ?cpropURI ?value
NOT EXISTS{
?i ?tpropURI ?value1 .
}
}UNION{
?i a ?tclassURI .
?i ?cpropURI ?value .
?i ?tpropURI "" .
}
FILTER (str(?value) = str(?condvalue)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 3: Syntax Violations (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||All instances with property values that have syntax violations
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:SyntaxRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}
</syntaxhighlight>
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 4: Conditional Syntax Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.
|-
|'''Input:'''||All conditional syntax rules(in OWL DL Design).
|-
|'''Output:'''||All instances of the subset of the class that have syntax violations.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ConditionalSyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
?i a ?tclassURI .
?i ?cpropURI ?condvalueplain .
?i ?tpropURI ?value .
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 5: Out of Range Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have property values out of a specified range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||All instances with out of range values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 6: Illegal Values (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.
|-
|'''Input:'''||All legal value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with values not listed in the trusted reference.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 7: Uniqueness Violations (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances with property values that are not unique although they should.
|-
|'''Input:'''||All unique value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}
GROUP BY ?dqr ?i
</syntaxhighlight>

[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 8: Functional Dependency Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?dvalue ?value
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 9: Functional Dependency Violations (2 Conditions, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
?dqr dqm:hasCondition2 ?cond2 .
?cond2 dqm:conditionalProperty ?cprop2 .
?cprop2 dqm:hasURI ?cpropreal2 .
?cond2 dqm:equals ?condvalue2 .
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?cpropURI2 ?condvalue2 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 10: Expired Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.
|-
|'''Output:'''||All instances with that are not valid anymore.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ExpiryRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}
</syntaxhighlight>

[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 11: Not Updated Values (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.
|-
|'''Output:'''||All instances with that are potentially outdated.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}
</syntaxhighlight>

[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

Data Quality Monitoring Reports

2011-10-20T20:40:51Z

Cfuerber:

'''This site is currently under construction!''' 
Problem reports are an excellent instrument to quickly identify instances with requirement violations.

==Prerequisites==
With the DQM-Vocabulary you can easily create problem reports. All you need is a SPARQL-Endpoint that supports [http://www.w3.org/TR/sparql11-query/ SPARQL 1.1] or SPARQL extensions that cover datatype conversion and conversion of strings to [[Wikipedia:URI|URIs]].
Before you can automatically analyze your data for requirement violations, you need to perform the following steps:

* [[Create Data Requirements#Define Tested Elements|Declare the elements of your ontology that shall be used in the DQM-Vocabulary]]
* [[Create Data Requirements]]

==Data Requirement Violation Reports with SPARQL==

Each type of data requirement has different problem reports, since the nature of data requirements differs. Moreover, the actual SPARQL query you can use depends on the design option you have chosen to make your ontology elements an instance of [[dqm:TestedClass]] and [[dqm:TestedProperty]] (See [[Create Data Requirements#Define Tested Elements]] for an explanation). In the following you find several SPARQL queries that make use of your data requirements to identify class instances with requirement violations.

===Example 1.1: Property Completeness Violations (OWL Full Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL Full Design)
|-
|'''Output:'''||All instances with missing values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
{
?i a ?tclass .
NOT EXISTS{
?i ?tprop ?value .
}
}UNION{
?i a ?tclass .
?i ?tprop "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule |Click here to learn how to define a data requirement for this report]]

===Example 1.2: Property Completeness Violations (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that miss a property or a property value.
|-
|'''Input:'''||All property completeness rules (in OWL DL Design)
|-
|'''Output:'''||All instances with missing values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:PropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
{
?i a ?tclassURI .
NOT EXISTS{
?i ?tpropURI ?value .
}
}UNION{
?i a ?tclassURI .
?i ?tpropURI "" .
}
}
</syntaxhighlight>
[[Create Data Requirements#Example 1: Property Completeness Rule|Click here to learn how to define a data requirement for this report]]

===Example 2: Conditional Property Completeness Violations (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances in a subset of a certain class that miss a property or a property value.
|-
|'''Input:'''||All conditional property completeness rules (in OWL DL Design)
|-
|'''Output:'''||Instances with missing values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?condvalue
WHERE {
?dqr a dqm:ConditionalPropertyCompletenessRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:requiredProperty "true"^^xsd:boolean .
?dqr dqm:requiredValue "true"^^xsd:boolean .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
{
?i a ?tclassURI .
?i ?cpropURI ?value
NOT EXISTS{
?i ?tpropURI ?value1 .
}
}UNION{
?i a ?tclassURI .
?i ?cpropURI ?value .
?i ?tpropURI "" .
}
FILTER (str(?value) = str(?condvalue)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 3: Syntax Rule Violations (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances of a certain class with property values that violate a syntax rule.
|-
|'''Input:'''||All syntax rules (in OWL DL Design)
|-
|'''Output:'''||All instances with property values that have syntax violations
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:SyntaxRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (!regex(str(?value), ?regex)) .
}
</syntaxhighlight>
[[Create Data Requirements#Example 3: Syntax Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 4: Conditional Syntax Rule Violations (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all syntax violations of property values in a subset of a certain class.
|-
|'''Input:'''||All conditional syntax rules(in OWL DL Design).
|-
|'''Output:'''||All instances of the subset of the class that have syntax violations.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ConditionalSyntaxRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:regex ?regex .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop .
?cprop dqm:hasURI ?cpropreal .
?cond1 dqm:equals ?condvalue .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal)) AS ?cpropURI) .
?i a ?tclassURI .
?i ?cpropURI ?condvalueplain .
?i ?tpropURI ?value .
FILTER (str(?condvalue) = str(?condvalueplain) && !regex(str(?value), ?regex)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 4: Conditional Syntax Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 5: Legal Value Range Rule (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have property values out of a specified range.
|-
|'''Input:'''||All legal value range rules (in OWL DL Design)
|-
|'''Output:'''||All instances with out of range values
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRangeRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
OPTIONAL{
?dqr dqm:upperLimit ?upperLimit .
}
OPTIONAL{
?dqr dqm:lowerLimit ?lowerLimit .
}
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
FILTER (STRDT(str(?value), xsd:float) > ?upperLimit ||
STRDT(str(?value), xsd:float) < ?lowerLimit) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 5: Legal Value Range Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 6: Legal Value Rule (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances that have illegal property values, i.e. values that are not listed in the trusted reference property.
|-
|'''Input:'''||All legal value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with values not listed in the trusted reference.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:LegalValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?dqr dqm:referenceClass ?rclass .
?dqr dqm:referenceProperty1 ?rprop .
?rclass dqm:hasURI ?rclassvalue .
?rprop dqm:hasURI ?rpropvalue .
BIND (IRI(str(?rpropvalue)) AS ?rpropURI) .
BIND (IRI(str(?rclassvalue)) AS ?rclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?value .
OPTIONAL {
?i2 a ?rclassURI .
?i2 ?rpropURI ?value1 .
FILTER (str(?value1) = str(?value)) .
} .
FILTER (!bound(?value1)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 6: Legal Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 7: Unique Value Rule (OWL DL Design)===
{|class="wikitable"
|'''Scenario:'''||Find all instances with property values that are not unique although they should.
|-
|'''Input:'''||All unique value rules (in OWL DL Design).
|-
|'''Output:'''||All instances with non-unique values of property that must have unique values for each instance.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UniqueValueRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
?i a ?tclassURI .
?i ?tpropURI ?uniqueValue1 .
?i2 ?tpropURI ?uniqueValue2 .
FILTER(?i!=?i2 && (str(?uniqueValue1) = str(?uniqueValue2)))
}
GROUP BY ?dqr ?i
</syntaxhighlight>

[[Create Data Requirements#Example 7: Unique Value Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 1 condition (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i ?dvalue ?value
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
MINUS{
?dqr dqm:hasCondition2 ?cond2 .
}
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all instances with inconsistent property values.
|-
|'''Input:'''||All functional dependency value rules with 2 conditions (in OWL DL Design).
|-
|'''Output:'''||All instances with inconsistent property values.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:FuncDepValueRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?dqr dqm:equals ?dvalue .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
?dqr dqm:hasCondition1 ?cond1 .
?cond1 dqm:conditionalProperty ?cprop1 .
?cprop1 dqm:hasURI ?cpropreal1 .
?cond1 dqm:equals ?condvalue1 .
?dqr dqm:hasCondition2 ?cond2 .
?cond2 dqm:conditionalProperty ?cprop2 .
?cprop2 dqm:hasURI ?cpropreal2 .
?cond2 dqm:equals ?condvalue2 .
MINUS{
?dqr dqm:hasCondition3 ?cond3 .
}
MINUS{
?dqr dqm:hasCondition4 ?cond4 .
}
MINUS{
?dqr dqm:hasCondition5 ?cond5 .
}
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND (IRI(str(?cpropreal1)) AS ?cpropURI1) .
BIND (IRI(str(?cpropreal2)) AS ?cpropURI2) .
?i a ?tclassURI .
?i ?cpropURI1 ?condvalue1 .
?i ?cpropURI2 ?condvalue2 .
?i ?tpropURI ?value .
FILTER (str(?dvalue) != str(?value)) .
}
</syntaxhighlight>

[[Create Data Requirements#Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 10: Expiry Rule (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All expiry rules (in OWL DL Design) and validity date of instances.
|-
|'''Output:'''||All instances with that are not valid anymore.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:ExpiryRule .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:testedClass ?tclass .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND ((now()) AS ?today)
?i a ?tclassURI .
?i ?tpropURI ?expiry .
FILTER(?expiry<?today)
}
</syntaxhighlight>

[[Create Data Requirements#Example 10: Expiry Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

===Example 11: Update Rule (OWL DL Design)===

{|class="wikitable"
|'''Scenario:'''||Find all outdated instances.
|-
|'''Input:'''||All update rules (in OWL DL Design) and timestamps of instances.
|-
|'''Output:'''||All instances with that are potentially outdated.
|}
<syntaxhighlight lang="sparql">
SELECT ?dqr ?i
WHERE {
?dqr a dqm:UpdateRule .
?dqr dqm:testedClass ?tclass .
?dqr dqm:testedProperty1 ?tprop .
?dqr dqm:expectedUpdateInterval ?duration .
?tclass dqm:hasURI ?tclassreal .
?tprop dqm:hasURI ?tpropreal .
BIND (IRI(str(?tpropreal)) AS ?tpropURI) .
BIND (IRI(str(?tclassreal)) AS ?tclassURI) .
BIND((dqf:requiredTimestamp(?duration)) AS ?reqTimestamp) .
?i a ?tclassURI .
?i ?tpropURI ?timestamp .
FILTER(?reqTimestamp>?timestamp)
}
</syntaxhighlight>

[[Create Data Requirements#Example 11: Update Rule (OWL DL)|Click here to learn how to define a data requirement for this report]]

Create Data Requirements

2011-10-20T20:40:26Z

Cfuerber: /* Example 11: Expiry Rule (OWL DL) */

'''This site is currently under construction!'''

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Sample Dataset used in Examples ==
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

==Syntax Of Examples==

The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].

== Examples of Data Requirements ==

===Example 1: Property Completeness Rule===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.1: Property Completeness Violations (OWL Full Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.2: Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2. Define the Conditional Property Completeness Rule'''

<pre>
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>
[[Generate_Problem_Reports#Example 2: Conditional Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<pre>
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</pre>

[[Generate_Problem_Reports#Example 3: Syntax Rule Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2.Define Conditional Syntax Rule'''
<pre>
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<pre>
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</pre>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<pre>
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<pre>
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</pre>

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<pre>
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</pre>

===Example 11: Update Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<pre>
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</pre>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.

Create Data Requirements

2011-10-20T20:34:23Z

Cfuerber: /* Example 1: PropertyCompletenessRule */

'''This site is currently under construction!'''

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Sample Dataset used in Examples ==
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

==Syntax Of Examples==

The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].

== Examples of Data Requirements ==

===Example 1: Property Completeness Rule===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.1: Property Completeness Violations (OWL Full Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.2: Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2. Define the Conditional Property Completeness Rule'''

<pre>
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>
[[Generate_Problem_Reports#Example 2: Conditional Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<pre>
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</pre>

[[Generate_Problem_Reports#Example 3: Syntax Rule Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2.Define Conditional Syntax Rule'''
<pre>
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<pre>
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</pre>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<pre>
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<pre>
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</pre>

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<pre>
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</pre>

===Example 11: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<pre>
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</pre>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.

Create Data Requirements

2011-10-20T20:32:28Z

Cfuerber: /* Examples of Data Requirements */

'''This site is currently under construction!'''

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Sample Dataset used in Examples ==
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

==Syntax Of Examples==

The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].

== Examples of Data Requirements ==

===Example 1: PropertyCompletenessRule===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.1: Property Completeness Violations (OWL Full Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.2: Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2. Define the Conditional Property Completeness Rule'''

<pre>
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>
[[Generate_Problem_Reports#Example 2: Conditional Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<pre>
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</pre>

[[Generate_Problem_Reports#Example 3: Syntax Rule Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2.Define Conditional Syntax Rule'''
<pre>
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<pre>
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</pre>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<pre>
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<pre>
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</pre>

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<pre>
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</pre>

===Example 11: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<pre>
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</pre>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.

Create Data Requirements

2011-10-20T20:31:15Z

Cfuerber: /* Example 1: PropertyCompletenessRule */

'''This site is currently under construction!'''

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Examples of Data Requirements ==

=== Sample Dataset used in Examples ===
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

===Syntax Of Examples===

The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].

===Example 1: PropertyCompletenessRule===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.1: Property Completeness Violations (OWL Full Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.2: Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2. Define the Conditional Property Completeness Rule'''

<pre>
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>
[[Generate_Problem_Reports#Example 2: Conditional Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<pre>
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</pre>

[[Generate_Problem_Reports#Example 3: Syntax Rule Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2.Define Conditional Syntax Rule'''
<pre>
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<pre>
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</pre>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<pre>
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<pre>
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</pre>

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<pre>
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</pre>

===Example 11: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<pre>
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</pre>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.

Create Data Requirements

2011-10-20T20:27:49Z

Cfuerber: /* Sample Dataset used in Examples */

'''This site is currently under construction!'''

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Examples of Data Requirements ==

=== Sample Dataset used in Examples ===
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

===Syntax Of Examples===

The following examples show instance data in [http://www.w3.org/TeamSubmission/turtle/ Turtle/Notation 3 syntax].

===Example 1: PropertyCompletenessRule===
A property completeness rule is a data requirement that specifies that a certain property and/or its value must exist in all instances of a certain class.

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.1: Property Completeness Violations (OWL Full Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.2: Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2. Define the Conditional Property Completeness Rule'''

<pre>
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>
[[Generate_Problem_Reports#Example 2: Conditional Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<pre>
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</pre>

[[Generate_Problem_Reports#Example 3: Syntax Rule Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2.Define Conditional Syntax Rule'''
<pre>
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<pre>
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</pre>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<pre>
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<pre>
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</pre>

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<pre>
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</pre>

===Example 11: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<pre>
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</pre>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.

Create Data Requirements

2011-10-20T20:23:38Z

Cfuerber: /* Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL) */

'''This site is currently under construction!'''

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Examples of Data Requirements ==

=== Sample Dataset used in Examples ===
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

===Example 1: PropertyCompletenessRule===
A property completeness rule is a data requirement that specifies that a certain property and/or its value must exist in all instances of a certain class.

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.1: Property Completeness Violations (OWL Full Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.2: Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2. Define the Conditional Property Completeness Rule'''

<pre>
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>
[[Generate_Problem_Reports#Example 2: Conditional Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<pre>
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</pre>

[[Generate_Problem_Reports#Example 3: Syntax Rule Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2.Define Conditional Syntax Rule'''
<pre>
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<pre>
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</pre>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<pre>
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<pre>
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</pre>

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<pre>
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</pre>

===Example 11: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<pre>
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</pre>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.

Create Data Requirements

2011-10-20T20:23:13Z

Cfuerber: /* Example 1: PropertyCompletenessRule */

'''This site is currently under construction!'''

__TOC__

== What are Data Requirements? ==

Data requirements are prescribed directives or consensual agreements that define the content and/or structure that constitute high quality data instances and values. Data requirements can thereby be stated by several different individuals or groups of individuals. Moreover, data requirments may also be based on laws, standards, or other directives. They may be agreed upon or contrary to each other.

However, data requirements are required as a prerequisite to measure data quality. Hence, they serve as a benchmark that defines the desired state of data. In the following, we describe how you can express your data requirements via the DQM-Vocabulary.

== Types of Data Requirements ==

Data requirements usually refer to different data items. When we look at a table we usually have at least four types of data items, (1) columns, (2) rows, (3) schemata, and (4) the table/spreadsheet itself.

[[File:Terminology.png|500px|Table to illustrate used terminology]]

In Semantic Web environments, we can compare columns to properties, rows to instances, schemata to ontologies, and tables to classes. Data requirements can usually be related to one of these elements. In particular, there are
# data requirements related to the values of a single property (column)
# data requirements related to the values of multiple properties within an instance (multiple columns in a row)
# data requirements related to the instances of a whole class (table)
# data requirements related to the ontology elements (schema)

With the DQM-Vocabulary, you can model the first three types of requirements. Schema/ontology requirements are currently not part of the vocabulary, but may be added in future releases. In the following, we explain how Property-, Multi-Property-, Class-, and Custom-Requirements can be modelled with the current version of the DQM-Vocabulary.

==Define Tested Elements==
Before you can use your data with the DQM-Vocabulary, you have to declare the elements of your ontology that shall be used in the DQM-Vocabulary. You have two options to do this with impact on decidablility of potential reasoning with your knowledge base:

=== Design Option 1: Classes and Properties as Instances (OWL Full) ===
Classes and properties that shall be tested for data requirement violations are defined as direct instances of the classes [[dqm:TestedClass]] or [[dqm:TestedProperty]].
<pre>
foo:MyClass a dqm:TestedClass
</pre>
<pre>
foo:MyProperty a dqm:TestedProperty
</pre>
'''Attention:''' This will make your knowledge base OWL Full which is potentially not useful if you plan to use reasoning.

=== Design Option 2: Mapping of Classes and Properties to new URIs (OWL DL) ===
Classes and properties that shall be tested for data requirement violations are mapped to new instances of the classes [[dqm:TestedClass]] and [[dqm:TestedProperty]].
<pre>
foo:Class_1 a dqm:TestedClass
dqm:hasURI "http://www.example.org/MyClass"^^xsd:anyURI
</pre>
<pre>
foo:Property_1 a dqm:TestedProperty
dqm:hasURI "http://www.example.org/MyProperty"^^xsd:anyURI
</pre>

== Examples of Data Requirements ==

=== Sample Dataset used in Examples ===
The following examples use classes and properties from our sample dataset as [[dqm:TestedClass]] and [[dqm:TestedProperties]]. Requirements specified in OWL DL will use the mapped instances, while requirements specified in OWL Full will use the original classes and properties. The data set contains the following classes:

{|class="wikitable"
!Original Class!!Mapped Instance
|-
|foo:Location||foo:Class_Location
|-
|}

...with the following datatype properties:
{| class="wikitable"
! Original Property !! Mapped Instance
|-
|foo:LOCID ||foo:Prop_Location_ID
|-
|foo:STREET||foo:Prop_Location_Street
|-
|foo:STREETNO||foo:Prop_Location_Streetno
|-
|foo:ZIP||foo:Prop_Location_ZIP
|-
|foo:CITY||foo:Prop_Location_City
|-
|foo:COUNTRY||foo:Prop_Location_Country
|-
|foo:STATE||foo:Prop_Location_State
|-
|foo:validThrough||foo:Prop_validThrough
|}

In order to apply the example to your own data, you need to exchange the sample classes and properties used in the data requirements by your own.

===Example 1: PropertyCompletenessRule===
A property completeness rule is a data requirement that specifies that a certain property and/or its value must exist in all instances of a certain class.

{|class="wikitable"
|'''Task:'''||Specify that a specific property and/or its values must exist for all instances of a specific class.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY must exist and have a value in all instances of the class foo:Location.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]] [[dqm:MissingProperty]] [[dqm:MissingValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

If you defined your data elements in OWL Full ([[#Option 1: Classes and Properties as Instances (OWL Full)|Option 1]]), then you can simply use the URIs of your ontology in the definition of the Property Completeness Rule as follows:

'''Definition in OWL Full'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass http://www.example.org/MyClass ;
dqm:testedProperty1 http://www.example.org/MyProperty ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.1: Property Completeness Violations (OWL Full Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" must exist in each instance. The property [[dqm:requiredValue]] specifies that a value must exist for property "MyProperty".

If you mapped your own ontology elements to new URIs ([[#Option 2: Mapping of Classes and Properties to new URIs (OWL DL)|Option 2]], OWL DL), then the following example will help you to define a Property Completeness Rule:

'''Definition in OWL-DL'''
<pre>
foo:PropertyCompletenessRule_1
a dqm:PropertyCompletenessRule ;
dqm:testedClass foo:Class_1 ;
dqm:testedProperty1 foo:Property_1 ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean .
</pre>
[[Generate_Problem_Reports#Example 1.2: Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

The property [[dqm:requiredProperty]] specifies that the property "MyProperty" which is mapped to "foo:Property_1" must exist in each instance of the class "MyClass" which is mapped to "foo:Class_1". The property [[dqm:requiredValue]] specifies that also a value must exist for property "foo:Property_1".

===Example 2: Conditional Property Completeness Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must exist if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must exist and have a value in all instances of the class foo:Location that have value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:MissingPropertyAndValue]]
|-
|'''Dimension:'''||[[dqm:PropertyCompleteness]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

To define this data requirement, you must perform the following two steps:

'''1. Define the Condition'''

<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2. Define the Conditional Property Completeness Rule'''

<pre>
foo:ConditionalPropertyCompletenessRule_State
a dqm:ConditionalPropertyCompletenessRule ;
rdfs:label "Conditional property completeness rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:requiredProperty "true"^^xsd:boolean ;
dqm:requiredValue "true"^^xsd:boolean ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>
[[Generate_Problem_Reports#Example 2: Conditional Property Completeness Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 3: Syntax Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax.
|-
|'''Notional Example:'''||In a location data set, the property foo:ZIP must contain values with exactly five digits.
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify syntax requirements by creating an instance of the class [[dqm:SyntaxRule]], e.g. as follows:

<pre>
foo:SyntaxRule_ZIP
a dqm:SyntaxRule ;
rdfs:label "Syntax rule ZIP"^^xsd:string ;
dqm:regex "^[0-9]{5}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Property_ZIP .
</pre>

[[Generate_Problem_Reports#Example 3: Syntax Rule Violations (OWL DL Design)|Click here to learn how to generate a problem report from this]] 

===Example 4: Conditional Syntax Rule (1 Condition, OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a specific property must obtain a specific syntax if another property obtains a specific value.
|-
|'''Notional Example:'''||In a location data set, the property foo:STATE must contain a value with two letters if the property foo:COUNTRY has the value "USA".
|-
|'''DQ-Problem:'''|| [[dqm:SyntaxViolation]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify a conditional syntax rule, you must perform the following steps:

'''1.Define Condition'''
<pre>
foo:Condition_USA
a dqm:Condition ;
rdfs:label "Condition USA"^^xsd:string ;
dqm:conditionalProperty foo:Prop_Location_Country ;
dqm:equals "USA"^^xsd:string .
</pre>

'''2.Define Conditional Syntax Rule'''
<pre>
foo:ConditionalSyntaxRule_State
a dqm:ConditionalSyntaxRule ;
rdfs:label "Conditional syntax rule State"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:regex "^[A-Z]{2}$"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 5: Legal Value Range Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify valid value ranges for properties that hold numeric values.
|-
|'''Notional Example:'''||In a product data set, the property foo:PRICE can never contain negative values.
|-
|'''DQ-Problem:'''|| [[dqm:OutOfRangeValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify a legal value range for a property by adding the following instance data to the class [[dqm:LegalValueRangeRule]]:

<pre>
foo:LegalValueRangeRule_Price
a dqm:LegalValueRangeRule ;
rdfs:label "Legal value range rule Price"^^xsd:string ;
dqm:lowerLimit "0.00"^^xsd:float ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_Price .
</pre>

The class [[dqm:LegalValueRangeRule]] has the special properties
* [[dqm:lowerLimit]]
* [[dqm:upperLimit]]

to specify the lowest and/or highest allowed value.

===Example 6: Legal Value Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify a reference property that holds the allowed values.
|-
|'''Notional Example:'''||In a location data set, the property foo:COUNTRY can only contain values of the trusted property foo:legalValue in the trusted class foo:LegalValueCountry.
|-
|'''DQ-Problem:'''|| [[dqm:IllegalValue]]
|-
|'''Dimension:'''||[[dqm:SyntacticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

In order to specify legal values for a specific property, you need to perform the following steps:

# Create a reference data set or use an existing data set that holds the legal values
# Specify the class and property which hold the legal values as [[dqm:TrustedClass]] and [[dqm:TrustedProperty]]
# Create an instance of [[dqm:LegalValueRule]], e.g. as follows:

<pre>
foo:LegalValueRule_Country
a dqm:LegalValueRule ;
rdfs:label "Legal value rule Country"^^xsd:string ;
dqm:referenceClass foo:TrustedClass_LegalValueCountry ;
dqm:referenceProperty1 foo:TrustedProperty_LegalValue ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 7: Unique Value Rule (OWL DL)===
{|class="wikitable"
|'''Task:'''||Specify that values of a property must be unique.
|-
|'''Notional Example:'''||In a location data set, the property foo:LOCID of class foo:Location must only contain unique values.
|-
|'''DQ-Problem:'''|| [[dqm:UniquenessViolation]]
|-
|'''Dimension:'''||[[dqm:PropertyUniqueness]]
|-
|'''Requirement Type:'''||[[dqm:PropertyRequirement]]
|}

You can specify that values of a property must be unique by creating an instance of the class [[dqm:UniqueValueRule]]:

<pre>
foo:UniqueValueRule_LOCID
a dqm:UniqueValueRule ;
rdfs:label "Unique value rule LOCID"^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_ID .
</pre>

===Example 8: Functional Dependency Value Rule (1 Condition, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second property has a certain value.
|-
|'''Notional Example:'''||In an address data set, the city name "New York" must always have the value "USA" for the property foo:COUNTRY.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between two property values, you must perform the following steps:

# Define a condition under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_1
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 1"^^xsd:string ;
dqm:equals "USA"^^xsd:string ;
dqm:hasCondition1 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" then the country must be
\"USA\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_Country .
</pre>

===Example 9: Functional Dependency Value Rule (2 Conditions, OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that a value of one property must obtain a specific value if a second and third property have specific values.
|-
|'''Notional Example:'''||In an address data set, the city "New York" in the country "USA" must always have the value "NY" for the property foo:STATE.
|-
|'''DQ-Problem:'''|| [[dqm:FunctionalDependencyViolation]]
|-
|'''Dimension:'''||[[dqm:SemanticAccuracy]]
|-
|'''Requirement Type:'''||[[dqm:MultiPropertyRequirement]]
|}

In order to specify the dependency between three property values, you must perform the following steps:

# Define both conditions under which a specific value is always required.
# Create an instance of the class [[dqm:FuncDepValueRule]], e.g. as follows:

<pre>
foo:FuncDepValueRule_2
a dqm:FuncDepValueRule ;
rdfs:label "Func dep value rule 2"^^xsd:string ;
dqm:equals "NY"^^xsd:string ;
dqm:hasCondition1 foo:Condition_USA ;
dqm:hasCondition2 foo:Condition_New_York ;
dqm:reqDescription "If the city value is \"New York\" and the country value is
\"USA\" then the state must be \"NY\"."^^xsd:string ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_State .
</pre>

===Example 10: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class expire.
|-
|'''Notional Example:'''||In a product data set, the class foo:Product has instances with product offerings that expire on a certain date which is specified via the property foo:validThrough.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify that instances of a class have an expiry date by creating an instance of class [[dqm:ExpiryRule]], e.g. as follows:

<pre>
foo:ExpiryRule_1
a dqm:ExpiryRule ;
rdfs:label "Expiry rule 1"^^xsd:string ;
dqm:testedClass foo:Class_Product ;
dqm:testedProperty1 foo:Prop_Product_validThrough .
</pre>

===Example 11: Expiry Rule (OWL DL)===

{|class="wikitable"
|'''Task:'''||Specify that instances of a specific class must be updated within a specified interval.
|-
|'''Notional Example:'''||In a location data set, the class foo:Location has instances with address data that have timestamps of the last update. The instances shall not be elder than 1 year 2 months 3 days 5 hours 20 minutes and 30.123 seconds.
|-
|'''DQ-Problem:'''|| [[dqm:OutdatedInstance]]
|-
|'''Dimension:'''||[[dqm:Timeliness]]
|-
|'''Requirement Type:'''||[[dqm:ClassRequirement]]
|}

You can specify a required update interval for instances of a specific property by creating an instance of class [[dqm:UpdateRule]], e.g. as follows:

<pre>
foo:UpdateRule_Location
a dqm:UpdateRule ;
rdfs:label "Update rule Location"^^xsd:string ;
dqm:expectedUpdateInterval "P1Y2M3DT5H20M30.123S"^^xsd:duration ;
dqm:testedClass foo:Class_Location ;
dqm:testedProperty1 foo:Prop_Location_timestamp .
</pre>

'''NOTE:''' The tested class must have a property that holds the time of the last update in order to be able to specify this requirement.