This page contains migration guides for migrating Stardog from one major version to another (e.g. 6.X to 7.Y)
<details open markdown="block"> <summary> Page Contents </summary> 1. TOC </details>There is a major change in Stardog 12 compared to Stardog 11:
The index.aggregate database option is deprecated and the aggregated (counting) indexes have been removed.
This decision was made to simplify the storage engine and improve write performance in multi-user environments. We have also determined
that the aggregated indexes were not widely used in practice.
The index.aggregate database option (with values On and Context) has been deprecated and the underlying aggregated index implementation has been removed. When upgrading from Stardog 11 to Stardog 12, any existing aggregated indexes will be disabled and removed automatically.
The aggregated indexes were previously used for:
This change does not affect query correctness or database operations. The primary impact is that exact triple counts are no longer maintained redundantly.
Upgrading from Stardog 11 to Stardog 12 requires an explicit upgrade step because of the index format changes. Follow these instructions:
server start command with the --upgrade argument, OR
b. Updating stardog.properties in STARDOG_HOME directory to have the upgrade.automatic = true entryThere is no separate migration or upgrade command required. The upgrade will automatically remove the deprecated aggregated indexes.
If you update stardog.properties to enable automatic upgrade you should remove the additional entry to avoid unintentional automated upgrades in the future.
The upgrade from Stardog 11 to 12 cannot be reversed. Make sure to create backups before upgrading.
Stardog 12 requires Java 21 for the CLI tools. If you are using the CLI to connect to a remote Stardog server, ensure Java 21 is installed. See CLI Installation for platform-specific installation instructions.
Stardog 12.0.0 adds a new permission model for stored queries (see below). This requires a manual migration of single-server and cluster instances that contain stored queries created using Stardog 11 or earlier to properly set them up as permission resources. The procedure is simple and consists of the following 4 steps:
stardog-admin --server https://<server>:5820 stored export <credentials> /path/to/output/file.stardog-admin --server https://<server>:5820 stored remove --all <credentials>stardog-admin --server https://<server>:5820 stored import --overwrite <credentials> /path/to/file/from/step/2There could be a complication at step 4 if some stored queries are only executable against databases that no longer exist in this Stardog instance. We recommend temporarily recreating such databases without data to be able to import the queries. Then the queries can either be updated or deleted, and the databases removed.
Starting with Stardog 12, the CLI tools are distributed separately from the server Docker image. The server continues to be deployed via Docker, while the CLI can be installed independently on any OS (Linux, Mac, Windows) using the provided packages. See CLI Installation for details.
You can also run CLI commands directly inside the Docker container without a separate CLI installation. See Docker-specific Operations for examples.
The Query Log Database is now enabled by default. This feature logs all SPARQL queries to a dedicated database (querylog) for analysis and monitoring purposes.
If you do not need query logging and want to avoid the associated overhead, you can disable it by setting query.log.enabled=false in your stardog.properties file. See the Query Log Database settings for all available configuration options.
The logging.access.type and logging.audit.type configuration properties have been removed. Access and audit logs are now always written in JSON format (JSON Lines, one entry per line).
If you had existing binary or text format log files, Stardog will automatically rotate them on startup (renaming them with a date suffix) and begin writing new JSON format logs. A warning will be logged if the old configuration specified a non-JSON type.
Stardog 12 changes the default authentication plugin for the BI/SQL server from mysql_native_password to caching_sha2_password. This provides improved security using SHA-256 based password hashing.
The new authentication method requires secure password transmission for the first connection via either SSL/TLS or RSA public key exchange. You may be affected if your BI Server is not configured with SSL/TLS, or if your clients are older than MySQL 8.0.
Migration Options:
sql.server.default.auth.plugin=mysql_native_password for compatibility with older clients (least secure)See Configuring the BI Server for detailed configuration instructions.
Stardog 12 introduces enhanced role-based access control for stored queries. Users can now share individual stored queries with specific users or roles and organize them in a hierarchical manner. These changes are backward-compatible with existing stored query configurations.
See Security for Stored Queries for details on the new permissions model.
Exporting database contents now requires the execute permission on the new db-export resource type. This affects the data export CLI command and the associated HTTP API endpoint. Users who previously had read access to a database will need to be granted this additional permission to export data.
Stardog 12 enforces authorization checks on all Knowledge Catalog operations. In previous versions, any authenticated user could perform catalog operations without additional permission checks. Users who previously had access to catalog operations will need to be granted the appropriate permissions. See Knowledge Catalog Permissions for details on the required permissions for each operation.
In Stardog 12, the storage location for Knowledge Catalog schema data has changed. For newly created catalog databases, the schema is now stored in the named graph tag:stardog:api:context:schema instead of the default graph.
Note for existing catalog databases: The schema is not automatically migrated to the new location during upgrade. To migrate to the new schema location:
Run the following command to populate the schema in the new named graph:
$ stardog-admin --server https://<server>:5820 catalog reload-model
Optionally, clear the old schema data from the default graph:
$ stardog query https://<server>:5820/catalog "CLEAR DEFAULT"
Stardog 12 introduces a new modeling resource type that provides fine-grained control over database options used by Designer for publishing models. The read permission allows users to view these options, and the write permission allows users to modify the following database options:
database.namespacesgraph.aliasesmedia.enabledreasoning.schema.graphsreasoning.schemassearch.enabledsearch.lexical.enabledsearch.semantic.enabledsearch.semantic.modelvirtual.transparencyvoicebox.enabledvoicebox.example.questionsThe existing metadata permission continues to allow modification of all database options. Use the modeling permission to grant users limited access to only the options relevant for data modeling workflows.
The original BITES (Blob Indexing and Text Enrichment with Semantics) document store implementation has been removed in Stardog 12. This legacy feature provided document storage and text extraction capabilities directly within Stardog.
For unstructured data processing, users should migrate to Using Unstructured Data with Voicebox, which provides enhanced capabilities for ingesting and querying documents from sources like Google Drive, Microsoft OneDrive, and local storage through Voicebox's conversational AI interface.
Stardog 12 introduces a new database option reasoning.domain.range.rule.optimization that is enabled by default. This optimization removes redundant type assertions from queries or rule bodies when types can be inferred from schema.org domainIncludes/rangeIncludes declarations on properties.
This optimization may cause reasoning query results to change after upgrading if your instance data does not conform to the domain/range declarations defined in the schema. Specifically, queries may return more results than before because certain type constraints are being removed from the query based on the assumption that the data already satisfies those constraints.
For example, suppose your schema declares the domain of property :worksFor to be :Person:
:Person a owl:Class .
:worksFor a owl:ObjectProperty ;
so:domainIncludes :Person .
and you have the following instance data:
:Alice a :Person ;
:worksFor :ACME ;
:Bob :worksFor :ACME .
and this query:
?x a :Person ; :worksFor ?y .
The optimization will remove the ?x a :Person pattern because it is implied by the domain declaration. As a result, both :Alice and :Bob will be returned in the results even though :Bob does not have a type declaration for :Person. For this reason, if you only query for ?x a :Person the instance :Bob will not be returned.
If you do not want this behavior after upgrading and your data does not fully conform to the schema's domain/range declarations, you can disable this optimization by setting:
reasoning.domain.range.rule.optimization=false
This can be set when creating the database or modified on an existing database using the db metadata set command.
For more details about how this optimization works and the differences between RDFS and schema.org vocabularies, see Domains and Ranges in the reasoning troubleshooting guide.
There are two major changes in Stardog 11 compared to Stardog 10:
Starting with Stardog 11 the only supported deployment of Stardog server is via Docker. This change has been done to provide a more secure and robust deployment means that is self-contained so there are less configuration steps for users.
Docker images can be used to deploy a standalone-server or an HA cluster within Kubernetes as before. Stardog CLI can still be installed on any OS (Linux, Mac, Windows) with Java 11. The CLI commands can be used to connect to a remote Stardog server running in Docker.
Stardog 11 uses a different indexing format for geospatial data (if the spatial.enabled property is set for a database). The format is more compact and enables faster spatial indexing. Also, Stardog 11 distinguishes data and object properties based on their usage in the graph. This provides the query optimizer with more information for query optimization purposes.
Both these changes require an upgrade of previously stored spatial indexes and graph statistics. No user action is needed: when Stardog 11 is started the first time on a home directory created with Stardog 10 (or earlier) it will automatically upgrade geospatial indexes and statistics to the latest version. This means it will take longer for the server to start first time. Once the indexes are upgrade subsequent restarts will not need additional time.
Downgrading from Stardog 11 to Stardog 10 would also require all the statistics and geospatial indexes to be recreated. It is recommended to create backups of Stardog 10 home directory before the upgrade so if downgrade is needed the backup copy can be used as the new home directory for Stardog 10.
Stardog 11 makes a minor change to the behavior of GROUP BY in SPARQL when WHERE pattern matches no results. The SPARQL 1.1 specification allows different interpretations of what the results should be in that corner case. Specifically, Stardog 10 (and earlier) returns the so-called empty group row when grouping is done over the empty input. The empty group row has empty values for GROUP BY variables and default values for aggregation results (e.g., 0 for COUNT). Stardog 11 returns the empty result set in such cases, which is more intuitive and is also consistent with other systems (see details at SPARQL Grouping and Aggregation with no matches).
Semantic search introduced in Stardog 9 (relying on external service) has been replaced with another implementation that stores semantic vectors in full-text search index. To migrate existing databases to this new implementation, follow these steps.
$ stardog-admin db offline -n dbname
$ stardog-admin metadata set -o search.enabled=true search.semantic.enabled=true search.semantic.model="djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2" -- dbname
$ stardog-admin db online -n dbname
There are two major changes in Stardog 9 compared to Stardog 8:
We will describe the necessary steps for upgrading from Stardog 7 or 8 to Stardog 9 in the following sections and explain any changes in other functionality.
You will need to ensure that Java 11 is installed in the environment Stardog 9 will run. Otherwise follow the instructions for your operating system to install Java and verify the version of Java using the command java -version.
The Docker image of Stardog 9 includes Java 11 so no further action is needed. However, The use of /bin/sh and some other commands will no longer work in the Stardog 9 docker container due to a switch from using CentOS to Ubuntu for the underlying image. Users should switch to using /bin/bash and other Ubuntu equivalents.
There is a change to the storage format for the system database that prevents reverting back to version 8.* once the upgrade to 9 has been made. The upgrade, once allowed, is completely automated and the system database will be updated automatically to the new version. Follow these instructions for the upgrade:
server start command with the --upgrade argument, OR
b. Updating stardog.properties in STARDOG_HOME directory to have the upgrade.automatic = true entryThere is no separate migration or upgrade command required. Upgrade is expected to complete very quickly regardless of the size of data in your databases because only the contents of the system database is affected,
If you update stardog.properties to enable automatic upgrade you should remove the additional entry to avoid unintentional automated upgrades in the future.
Stardog 9 will not automatically migrate home directories created before version 7. If you would like to upgrade from an earlier version please follow the instructions for migrating to Stardog 7 first and then upgrade to Stardog 9.
Stardog 9 introduces a new inference engine named Stride. See the documentation for the details about this engine. The new inference engine will not take affect unless the reasoning.stride database option is set to true.
There are no changes to the existing reasoning behavior except the DL reasoning type that was already deprecated in previous versions has been completely removed in Stardog 9. If you have set the configuration option reasoning.type=DL those databases will be set to offline mode automatically after upgrading to Stardog 9. The configuration option should be set to a different type before the database can be brought online again.
Stardog provides the capability to auto generate GraphQL schemas or BI/SQL schema mappings. Schema generation can either use OWL declarations or SHACL shape definitions. The database configuration options graphql.auto.schema.source and sql.schema.auto.source determine which source will be used. If OWL is used for schema generation source Stardog 9 will not use the reasoner by default for schema generation. This change has several implications:
so:domainIncludes and so:rangeIncludes from the schema.org namespace will be treated similar to rdfs:domain and rdfs:range for creating attributes and columns.If you would like to generate the schemas with inference then you can use the following command to generate the schema manually and register it with Stardog:
$ stardog data model --reasoning --input owl --output sql DB
Please refer to the BI/SQL and GraphQL sections for the details of registering schemas manually.
Stardog 9 introduces new SPARQL extensions for performing data quality validation with SHACL. THe details of this new capability are explained in {Data Quality Constraints section](../data-quality-constraints). Before Stardog 9 the only way to perform data validation was to use the API or the CLI. These methods still work in Stardog 9 and the validation results should not change as a result to upgrading to Stardog 9. The data validation API in Stardog 9 is a simple wrapper to generate SPARQL VALIDATE queries but the API behavior is backward-compatible.
Before Stardog 9 it was possible to run a Stardog server embedded within the application JVM and connect to the server without going through the HTTP layer. There were several shortcomings of this capability and it was clearly stated not be a production feature. Most notably, embedded deployments suffered from server stability issues and several capabilities behaved different compared to standard server deployments. For this reason, the capability to run Stardog in embedded mode has been completely removed in Stardog 9. The main use case for embedded server was simpler setups in unit tests. Please see the Stardog API examples for how to deploy a Stardog server within a JVM and communicate via HTTP.
When Stardog server is started for the very first time with an empty STARDOG_HOME directory it creates an admin user. Prior to version 9.0 another user named anonymous was created automatically that would have read access to any resource on the server. Due to security complications caused by this setup Stardog 9.0 will not create the anonymous user by default. If stardog.properties configuration file contains the setting create.anonymous.user=true then the anonymous user will be created as in earlier versions. Note that, anonymous user has already been created then upgrading to Stardog 9 will not cause the user to be dropped.
There is a change to the storage format for data sources that prevents reverting back to version 8.0.1 once the upgrade to 8.1.0 has been made. Please backup your system database prior to upgrading to this release.
There are no major changes to the storage format in Stardog 8 compared to Stardog 7 so there are no migration steps required. You can start a Stardog 8 server using a Stardog home directory created by Stardog 7. The deprecated configuration options will be removed automatically when Stardog 8 starts the first time which should not take any noticeable amount of time. Due to these changes it is not recommended downgrading to Stardog 7 for that same home directory. As always, it is recommended to create a backup of your home directory before changing versions as an additional precaution.
Several features that were deprecated in Stardog 7 are removed in Stardog 8. The migration instructions for these specific features are described below.
Stardog 8 only supports constraints written in SHACL and removes the support for constraints written in OWL completely. As a consequence, there is no icv add command anymore. SHACL constraints can be added to a database using the regular data add command instead.
There is a utility class that can automatically translate some of the OWL constraints to semantically equivalent SHACL constraints. The following snippet shows how this utility can be used programmatically:
// import com.complexible.stardog.icv.ShaclGenerator;
Set<Statement> input = RDFParsers.read(Paths.get("path-to-OWL-file"));
List<Shape> shapes = new ShaclGenerator().convertStatements(input);
ShaclWriter writer = new ShaclWriter();
shapes.forEach(writer::writeShape);
NamespacesImpl namespaces = new NamespacesImpl(Namespaces.DEFAULT);
namespaces.add("sh", SHACL.NS);
RDFWriters.write(System.out, RDFFormats.PRETTY_TURTLE, writer.getGraph().graph(), namespaces);
Stardog 8 does not have any built-in archetypes. If you would like to use the archetypes, follow the instructions described in the Stardog Archetype Repository.
Stardog 7 introduces a new storage engine and snapshot isolation for concurrent transactions. This section provides an overview of those changes and how they affect users and programs written against previous versions.
The new storage engine in Stardog 7 introduces a completely new disk index format and databases created by previous versions of Stardog must be migrated in order to work with Stardog 7. There is a dedicated CLI command for migrating the contents of an existing Stardog home directory (i.e., all of the databases in a multi-tenant system).
The following instructions are for migrating all the databases in an existing STARDOG_HOME directory. Instead of migrating all the databases you can start with a new empty home directory and restore select databases using backups created by Stardog versions 4 or 5. If you use the following instructions with very large databases then you should increase the memory settings by setting the environment variable STARDOG_SERVER_JAVA_ARGS.
The steps for a single server migration:
NEW_HOME)NEW_HOMEcd to where you've installed Stardog 7# OLD_HOME is the STARDOG_HOME before you start the migration
$ stardog-admin server migrate OLD_HOME NEW_HOME
STARDOG_HOME (in your .bashrc profile or otherwise) to be equal to NEW_HOME.The command will migrate the contents of the each database along with the system database that contains users, roles, permissions, and other metadata. Progress for the migration will be printed to STDOUT and can take a significant amount of time if you have large databases. The stardog.properties (if it exists) file will not be copied automatically. See Disk Usage and Layout for changes to the configuration options and other information.
The migration process for Stardog running in Docker is effectively the same with a couple of Docker-specific differences.
NEW_HOME).NEW_HOME# OLD_HOME is the STARDOG_HOME before you start the migration
$ docker run -v <path to NEW_HOME>:/var/opt/stardog -v <path to OLD_HOME>:/old_stardog \
--entrypoint /bin/bash -it stardog-eps-docker.jfrog.io/stardog:6.0.0-alpha
$ stardog-admin server migrate /old_stardog /var/opt/stardog
STARDOG_HOME (in your bashrc profile or otherwise) to be equal to NEW_HOME.The migration steps for the cluster:
stardog-admin zk clearThe layout of data in Stardog 7 home directory is different than in all previous versions. Previously the data stored in a database was stored under a directory with the name of the database. In Stardog 7 the data for all databases is stored in a directory named data in the home directory. The database directories still exist but they contain only index metadata along with search and spatial index if those features are enabled.
The disk usage requirements for Stardog 7 are higher than Stardog 6. The actual difference will depend on the characteristics of your data, but you should expect to see 20% to 30% increase in disk usage. Similar to Stardog 6, the disk usage of bulk loaded databases, e.g. when data is loaded by the db create command, will be lower than the disk usage when the same data is added incrementally, that is, in smaller transactions over time.
The web console, which had been deprecated in Stardog 6, has been removed entirely from Stardog 7. We encourage you to use Stardog Studio instead.
Stardog 7 no longer supports in-memory databases. If keeping all data in memory is desired, we recommend placing the home directory on a RAM disk and create databases in the usual way.
Stardog 7 uses a new storage engine (RocksDB) which is a native library. No changes to the Java JVM memory settings are required, as Stardog will allocate memory to the storage engine from its off-heap pool. As with Stardog 6, users provide limits for the Java heap memory (-Xmx option) and the off-heap memory (-XX:MaxDirectMemorySize option). See Memory Usage for details.
Similar to Stardog 6, Stardog 7 performance degrades over time as the database is updated with transactions. The disk usage will continue to increase and data deleted by transactions will not be removed from disk. The existing db optimize can be used to perform index compaction on disk to improve the performance of reads and writes. The optimize command now provides additional options for the administrators to instruct which exact optimization steps to perform.
All server and database options and their meanings are unchanged in Stardog 7, with the following exceptions:
index.differential and index.writer. Stardog 7 has a new mechanism which replaces the previous implementation of Differential Indexes and Read-Your-Writes so these options are ignored.transaction.isolation needs to be set to SERIALIZABLE for ICV Guard Mode in order to ensure data integrity w.r.t. the constraints.#Stardog 7 uses a multi-versioned concurrency control (MVCC) model providing lock-free transactions with snapshot isolation guarantees. Stardog 6 provided a weaker snapshot isolation mechanism that required writers to acquire locks that sometimes blocked other transactions for a very long time, which is no longer the case. As a result, the performance of concurrent updates is greatly improved in Stardog 7, especially in the cluster setting.
This option provides the best write throughput performance but it also means Stardog cannot maintain the aggregate indexes it otherwise uses for statistics and answering some queries. For this reason, the database option index.aggregate is set to off in this mode.
There are two different modes for the MVCC transactions based on how conflicting changes made by two concurrent transactions will be handled by setting the transaction.write.conflict.strategy database option.
This is the default behavior (transaction.write.conflict.strategy=last_commit_wins) where the change made by the last committed transaction will be accepted. If two concurrent transactions try to add or remove the same quad the change made by the transaction last committed will be accepted while the other change is silently ignored. This is similar to Stardog 6 behavior which uses locks to achieve the same effect in a less efficient way.
This also means Stardog cannot track the exact size of the database without introducing additional overhead. In this mode, when you ask for the size of the database using the data size CLI command or Connection.size() API call you will get an approximate number. For example, if you add a quad that already exists in the database it might be double counted. Stardog will periodically update this number to be accurate but the accuracy is not guaranteed in general. The option to retrieve the exact size of the database is provided both in the CLI and the Java API but it will require scanning the contents of whole database which might be slow for large databases.
In this mode (transaction.write.conflict.strategy=abort_on_conflict), if two concurrent transactions try to add or remove the same quad, one of the transactions will be aborted with a transaction conflict. The client then should decide if conflicted transactions should be retried or aborted. This check introduces additional overhead for write transactions but makes it possible to maintain additional indexes and provide exact size information by setting the option index.aggregate to on.
Databases with Edge properties (a.k.a. RDF*) also require this mode to be set.
The index.aggregate option is deprecated as of Stardog 12 and aggregated indexes have been removed. See Migrating to Stardog 12 for details.
You may want to do additional configuration for two features added in Stardog 7. Read more about those here:
There are two major changes to take account of.
First, the primary incompatible change in Stardog 6 is a new core API, called Stark, which replaces RDF4j/Sesame as the core API around graph concepts. Additional information about that change is detailed below.
Second, the web console is DEPRECATED. It is still available in Stardog 6, but it is NOT supported. We encourage you to use Stardog Studio instead.
The first thing you might notice is some different naming conventions than traditional Java libraries. Most notably, the Java Bean-style conventions of get and set prefixes are abandoned in favor of shorter, more concise method names. Similarly, you'll notice exceptions are not post fixed with Exception, and are instead MalformedQuery or InvalidRDF. We don't think the Exception postfix adds anything; it's clear from usage that it's an Exception, no need to add noise to the name.
Additionally, you will not find null returned by any method in Stark. If it's the case that there is no return value, you get an Optional instead of null. This includes the optional context of a Statement; instead of using null to denote the default context, there's a specific constant to indicate this, namely Values#DEFAULT_GRAPH and utility methods on Values for checking if a Value or Statement corresponds to the default graph. If you're using an IDE that will leverage the JSR-305 annotations, @Nullable and @Nonnull, we've used these throughout the interface to document the behavior and you should see warnings if you're mis-using the API.
There's no longer a Graph class, so for cases where it's appropriate to return a collection of Statement, such as the result of parsing a file, we're simply using Set<Statement>. If you need to select subsets of Statement objects, such as all of the rdf:type assertions, there are utility methods provided from Graphs and Statements, or you can simply get a Stream from the Set and do the filting like you would with any other Collection.
Many of the core APIs have been cleaned up from their original counterparts. For example, Literal was trimmed down to just the basics, and if you need to get its value as a different type, like an int, there are static methods available from the Literal class.
In addition to the changes already mentioned, one thing to look out for is Value#stringValue on the older, Sesame based API. It returned the label of a Literal, the ID of a BNode and an IRI as a String. Generally, the correct replacement this behavior is Literal#str, but in some usages, using toString is sufficient. Value#toString in STARK returns the complete value of the Value object, eg, for a Literal it includes the lang/datatype, whereas stringValue did not.
This is a list of commonly used classes from the previous API, and their new counterparts:
| Sesame/RDF4J | Stark |
|---|---|
| ModelIO | RdfWriters, RdfParsers |
| TupleQueryResult | SelectQueryResult |
| Graph | java.util.Set |
| QueryResultIO | QueryResultWriters, QueryResultParsers |
| RDFFormat | RDFFormats |
The IRIs used to assess the quality of machine learning models have been renamed as follows:
| Stardog 5 | Stardog 6 |
|---|---|
| spa:validation | spa:evaluation |
| spa:validationMetric | spa:evaluationMetric |
| spa:validationScore | spa:evaluationScore |
See the examples in Automatic Evaluation section about the usage of these terms.