SHACL

SHACL#

This chapter is a short introduction to SHACL (Shapes Constraint Language) using rudof.

Preliminaries: install and configure rudof#

%pip install -q "pyrudof>=0.3.22"

Note: you may need to restart the kernel to use updated packages.

from pyrudof import Rudof

rudof = Rudof()

Shapes that are themselves RDF#

SHACL makes two different choices from ShEx:

  • Shapes are written with the sh: vocabulary, in Turtle or any other RDF syntax. They can be stored, queried and validated like any other data.

  • Because a shape is a node, it can carry its own targets: sh:targetClass, sh:targetNode, sh:targetSubjectsOf and sh:targetObjectsOf say which nodes the shape applies to. There is no separate shape map.

Let’s read some data:

rudof.read_data("""
prefix :       <http://example.org/>
prefix xsd:    <http://www.w3.org/2001/XMLSchema#>
prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs:   <http://www.w3.org/2000/01/rdf-schema#>

:timbl  rdf:type    :Human ;
        :name       "Tim Berners-Lee" ;
        :birthPlace :london ;
        :birthDate  "1955-06-08"^^xsd:date ;
        :employer   :CERN ;
        :knows      _:1 .
:london rdf:type    :city, :metropolis ;
        :country    :UK .
:CERN   rdf:type    :Organization .
_:1     :birthPlace :Spain .
""")

And a shapes graph describing it:

rudof.read_shacl("""
prefix :       <http://example.org/>
prefix sh:     <http://www.w3.org/ns/shacl#>
prefix xsd:    <http://www.w3.org/2001/XMLSchema#>

:Person a sh:NodeShape ;
 sh:targetClass :Human ;
 sh:property [ sh:path :name ;
               sh:datatype xsd:string ;
               sh:minCount 1 ; sh:maxCount 1 ] ;
 sh:property [ sh:path :birthDate ;
               sh:datatype xsd:date ;
               sh:maxCount 1 ] ;
 sh:property [ sh:path :birthPlace ;
               sh:node :Place ; sh:maxCount 1 ] ;
 sh:property [ sh:path :employer ;
               sh:node :Organization ] .

:Place a sh:NodeShape .
:Organization a sh:NodeShape .
""")

:Person is a node shape: it applies to a node as a whole. Its sh:targetClass :Human says it applies to every node typed :Human, which is what makes validation self-directing. Each sh:property is a property shape, describing the values reachable along one sh:path:

  • sh:datatype, sh:minCount and sh:maxCount are the counterparts of ShEx’s datatype constraints and cardinalities.

  • sh:node is the counterpart of ShEx’s @ reference: the value must itself conform to the named shape.

Validating#

report = rudof.validate_shacl()

print("conforms:", report.conforms)
print("violations:", len(report))

conforms: True
violations: 0

validate_shacl returns a ShaclValidationReport. This data conforms, so there is nothing to look at yet. Let’s add some nodes that break the shapes: :wrong1 has two birth dates where at most one is allowed, and :wrong2 has a :name that is an integer rather than a string.

rudof.read_data("""
prefix :    <http://example.org/>
prefix xsd: <http://www.w3.org/2001/XMLSchema#>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

:wrong1 rdf:type    :Human ;
        :name       "John" ;
        :birthDate  "1955-06-08"^^xsd:date, "1956-06-08"^^xsd:date .
:wrong2 rdf:type    :Human ;
        :name       23 ;
        :birthDate  "1955-06-08"^^xsd:date .
""", merge=True)

report = rudof.validate_shacl()
print("conforms:", report.conforms)
print("violations:", len(report))

conforms: False
violations: 2

Each entry in the report is a validation result, and carries the pieces you need to locate and explain the failure:

Field

Meaning

focus_node

the node that was being validated

path

the property path whose values broke the constraint

value

the offending value, when there is a single one

source_shape

the shape that raised it

constraint_component

which constraint: sh:MaxCountConstraintComponent, …

severity

sh:Violation, sh:Warning or sh:Info

message

a human-readable explanation

 
for entry in report:
    print(f"{entry.focus_node}")
    print(f"  path:       {entry.path}")
    print(f"  value:      {entry.value}")
    print(f"  constraint: {entry.constraint_component}")
    print(f"  severity:   {entry.severity}")
    print(f"  message:    {entry.message}")
    print()

http://example.org/wrong1
  path:       http://example.org/birthDate
  value:      None
  constraint: http://www.w3.org/ns/shacl#MaxCountConstraintComponent
  severity:   Violation
  message:    "MaxCount(1) not satisfied"

http://example.org/wrong2
  path:       http://example.org/name
  value:      "23"^^xsd:integer
  constraint: http://www.w3.org/ns/shacl#DatatypeConstraintComponent
  severity:   Violation
  message:    "Expected Datatype: xsd:string"

Rendering the report#

serialize_shacl_validation_results renders the current results. ResultShaclValidationFormat chooses the rendering and ShaclValidationSortMode the ordering:

from pyrudof import ResultShaclValidationFormat, ShaclValidationSortMode

print([str(f) for f in ResultShaclValidationFormat.all()])
print([str(m) for m in ShaclValidationSortMode.all()])

['Details', 'Turtle', 'NTriples', 'RdfXml', 'TriG', 'N3', 'NQuads', 'Minimal', 'Compact', 'Json', 'Csv']
['Severity', 'Node', 'Component', 'Value', 'Path', 'SourceShape', 'Details']

print(rudof.serialize_shacl_validation_results(ResultShaclValidationFormat.Minimal))

Does not conform, 2 violations, 0 warnings

import re

# rudof's table renderings are written for a terminal, and make the severity IRI a
# clickable hyperlink using an escape sequence a notebook does not understand. Strip
# those; the colours are left alone, since those a notebook renders fine.
HYPERLINK = re.compile("\x1b\\]8;;.*?\x1b\\\\")


def as_text(rendered):
    return HYPERLINK.sub("", rendered)


print(as_text(rudof.serialize_shacl_validation_results(
    ResultShaclValidationFormat.Compact,
    ShaclValidationSortMode.Node,
)))

╭──────────────┬─────────┬────────────────────────────────┬────────────┬───────────────────┬────────────────────────────────────╮
│ Severity     │ Node    │ Component                      │ Path       │ Value             │ Source shape                       │
├──────────────┼─────────┼────────────────────────────────┼────────────┼───────────────────┼────────────────────────────────────┤
│ sh:Violation │ :wrong1 │ sh:MaxCountConstraintComponent │ :birthDate │                   │ _:bbc18e6eaece948f48dd5841b6aff7a6 │
├──────────────┼─────────┼────────────────────────────────┼────────────┼───────────────────┼────────────────────────────────────┤
│ sh:Violation │ :wrong2 │ sh:DatatypeConstraintComponent │ :name      │ "23"^^xsd:integer │ _:b5c526f7ad2d129b4f9ed712f10a3b77 │
╰──────────────┴─────────┴────────────────────────────────┴────────────┴───────────────────┴────────────────────────────────────╯

The RDF renderings deserve a special mention. SHACL defines the validation report itself as RDF, using the sh:ValidationReport and sh:ValidationResult vocabulary. So a report can be stored in a graph, queried with SPARQL, or validated against a shape in turn:

print(rudof.serialize_shacl_validation_results(ResultShaclValidationFormat.Turtle))

@prefix sh: <http://www.w3.org/ns/shacl#> .
_:b0 sh:result _:b1 , _:b2 ;
	a sh:ValidationReport ;
	sh:conforms false .
_:b1 sh:sourceShape _:b3 ;
	sh:focusNode <http://example.org/wrong1> ;
	a sh:ValidationResult ;
	sh:resultPath <http://example.org/birthDate> ;
	sh:resultSeverity sh:Violation ;
	sh:sourceConstraintComponent sh:MaxCountConstraintComponent ;
	sh:resultMessage "MaxCount(1) not satisfied" .
_:b2 sh:sourceShape _:b4 ;
	sh:focusNode <http://example.org/wrong2> ;
	a sh:ValidationResult ;
	sh:resultPath <http://example.org/name> ;
	sh:resultSeverity sh:Violation ;
	sh:sourceConstraintComponent sh:DatatypeConstraintComponent ;
	sh:value 23 ;
	sh:resultMessage "Expected Datatype: xsd:string" .

Validation modes#

rudof ships more than one SHACL engine, selected with ShaclValidationMode. Native runs rudof’s own implementation; the SPARQL-based mode expresses the constraints as SPARQL queries instead.

from pyrudof import ShaclValidationMode

print([str(m) for m in ShaclValidationMode.all()])

['Native', 'Sparql']

report = rudof.validate_shacl(ShaclValidationMode.Native)
print("conforms:", report.conforms, "| violations:", len(report))

conforms: False | violations: 2

Shapes and data in one graph#

Because shapes are RDF, a single file can perfectly well hold both the shapes and the instances they describe. Calling read_shacl() with no input tells rudof to take the shapes graph from the data already loaded in the session:

with Rudof() as session:
    session.read_data("""
    prefix :    <http://example.org/>
    prefix sh:  <http://www.w3.org/ns/shacl#>
    prefix xsd: <http://www.w3.org/2001/XMLSchema#>

    :PersonShape a sh:NodeShape ;
      sh:targetClass :Person ;
      sh:property [ sh:path :name ; sh:datatype xsd:string ; sh:minCount 1 ] .

    :alice a :Person ; :name "Alice" .
    :bob   a :Person .
    """)

    session.read_shacl()   # no input: the shapes come from the loaded data

    report = session.validate_shacl(ShaclValidationMode.Native)
    print("conforms:", report.conforms)
    for entry in report:
        print(f"  {entry.focus_node} — {entry.message}")

conforms: False
  http://example.org/bob — "MinCount(1) not satisfied"

ShapesGraphSource is the enum that names this choice explicitly, for configurations that need to state where the shapes come from rather than infer it:

from pyrudof import ShapesGraphSource

print([str(s) for s in ShapesGraphSource.all()])

['CurrentData', 'CurrentSchema']

Serializing a shapes graph#

A loaded shapes graph can be written back out in any RDF syntax, which is how you move shapes between tools that prefer different formats:

from pyrudof import ShaclFormat

print([str(f) for f in ShaclFormat.all()])

['Internal', 'Turtle', 'NTriples', 'RdfXml', 'TriG', 'N3', 'NQuads', 'JsonLd', 'Json']

with Rudof() as session:
    session.read_shacl("""
    prefix :   <http://example.org/>
    prefix sh: <http://www.w3.org/ns/shacl#>

    :PersonShape a sh:NodeShape ;
      sh:targetClass :Person ;
      sh:property [ sh:path :name ; sh:minCount 1 ] .
    """)

    print(session.serialize_shacl(ShaclFormat.NTriples))

<http://example.org/PersonShape> <http://www.w3.org/ns/shacl#targetClass> <http://example.org/Person> .
<http://example.org/PersonShape> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/ns/shacl#NodeShape> .
<http://example.org/PersonShape> <http://www.w3.org/ns/shacl#property> _:b0 .
_:b0 <http://www.w3.org/ns/shacl#path> <http://example.org/name> .
_:b0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/ns/shacl#PropertyShape> .
_:b0 <http://www.w3.org/ns/shacl#minCount> "1"^^<http://www.w3.org/2001/XMLSchema#integer> .

References#