Schema upgrade

From DataObjects.Net Wiki

Content of this section is outdated. Please refer to new DataObjects.Net Manual for actual information.

Schema upgrade overview

Every time new Domain is built, schema upgrade is performed. Schema upgrade can be performed fully automatically, but in complex cases it requires assistance from developer - you can provide upgrade hints and/or write custom upgrade handler to implement complex schema upgrade.

There are four different ways to upgrade a database schema, all of them are listed in DomainUpgradeMode enumeration:

Validate: No upgrade is performed, but existing schema is checked for compability with schema deduced from your persistent types. Data is never modified in this mode.
Recreate: All existing objects in schema are dropped, all required objects are created, so the data is completely destroyed.
Perform: Schema upgrade is performed. Unused tables and columns in the database are removed, required tables and columns are created. Data is preserved, but some columns and tables may be lost. Upgrade hints are taken into account.
PerformSafely: Much like Perform, but any actions leading to data or precision lost are never performed to the database unless all of them are explicitly enabled by upgrade hints. If upgrade can not be performed - for example, when some column must be deleted, and no hint enabling this operation is provided - an exception is thrown before any modifications are made to the database. This is the default upgrade mode.

You can specify upgrade mode in domain configuration.

Upgrade steps in Perform \ PerformSafely modes

Validating the possibility of upgrade: we query upgrade handlers of each assembly with persistent types if they're ready to upgrade their own part of schema from the previous version. So we support explicit numbering of schema versions for each persistent assembly. By default these handlers return version specified in [AssemblyInfo] attribute.
Collecting upgrade hints: upgrade handlers are queried for upgrade hints (UpgradeHint ancestors). Such hints establish associations between old and new types and fields, if they were renamed, as well as provide additional instructions to schema comparison and upgrade layer.
Building upgrade action sequence: schema comparison and upgrade layer compares old and new schema taking upgrade hints into account, and builds a sequence of schema upgrade actions that must be performed.
Checking upgrade safety: if you're using PerformSafely upgrade mode, upgrade action sequence is checked for presence of any actions that may lead to data loss. If such actions are found, and at least one of them isn't explicitly enabled by the upgrade hint, the upgrade fails.
Performing schema upgrade: the produced action sequence is translated into database-specific command sequence, and these commands are executed against the database.
Performing metadata upgrade: as it was mentioned, we store metadata allowing schema upgrade layer to operate more intelligently. In particular, we store:

Persistent model the schema is build for. It allows to take into account changes made to it and define upgrade hints on type level rather than on schema level.
Persistent assembly versions - they're used during validation of possibility to upgrade and provide additional information for assembly upgrade handlers.

Performing data upgrade: OnUpgrade method of each assembly upgrade handler is invoked to upgrade the data, if this is necessary. Recycled types can be used there. Recycled types are types that must be destroyed at the end of upgrade, but that must be available on this stage.
Destruction of recycled types: actually, one more schema upgrade leading to destruction of all recycled types there.

Implementing schema upgrade

Specifying name and version of your assembly

DataObjects.Net upgrade mechanism does not rely on regular assembly name and version. Instead it provides [AssemblyInfo] attribute that can be used to specify name and version of assembly exclusively for upgrade purposes. You should apply this attribute to each assembly with persistent types from the very first version of your code to make further upgrades possible. Usage of AssemblyInfoAttribute is simple. The following example is pretty self-describing.

Using AssemblyInfoAttribute:

[assembly: AssemblyInfo("Xtensive.Storage.Samples.Upgrade", "1.0")]

Please, note that version argument is a string. No special version string format is required.

Writing custom upgrade handler

Firstly, to perform upgrade you need to write a custom upgrade handler. Upgrade handler is a class that inherits from UpgradeHandler class. Exactly one upgrade handler is required for each assembly that contain persistent types. If you do not specify upgrade handler default one is used. Upgrade handler should be placed in the same assembly it is written for and should have parameterless constructor. Upgrade mechanism will automatically detect upgrade handlers and instantiate them.

UpgradeHandler base class contains many useful methods to override, but the most important are:

CanUpgradeFrom Called to check whenever this upgrade handler is suitable for upgrading assembly from the specified version. This is exactly the same version that was used in AssemblyInfoAttribute.

AddUpgradeHints Called to demand upgrade hints from upgrade handler.

OnUpgrade Called to perform custom upgrade actions.

Using upgrade hints

Upgrade hints are one of the most important features of DataObjects.Net upgrade mechanism. Upgrade hint is a simple object that describe additional information about model changes to aid upgrader perform its tasks. Upgrade via upgrade hints sometimes called "semi-automatic upgrade".

Here are the most common upgrade hints:

RenameTypeHint

Describes a renaming of a persistent type. You should create this hint when you rename entities or structures. Let's look how this hint is used. In first version of model we had Category entity, but in second version we decided to rename it to ProductGroup.

Using RenameTypeHint:

new RenameTypeHint(
  "Xtensive.Storage.Samples.Upgrade.Model.Category",
  typeof (ProductGroup));

As you can see the first argument of the constructor is a string, that represents a full name of the type being renamed. The second argument is a corresponding type from the new version of the model.

RenameFieldHint

Describes a renaming of a persistent field. You should create this hint when you rename a persistent field. As with previous example we renamed Category to ProductGroup. Also, we want to rename a field in Product entity to reflect this change. Let's rename Category field to Group.

Using RenameFieldHint:

new RenameFieldHint(typeof (Product), "Category", "Group"));

Again, creating of a hint is very simple. We specify the type that own a renamed field, old field name and new field name. It's important to note, that if you rename both type and field in this type, you should specify a new type in RenameFieldHint.

CopyDataHint

Describes a moving of a persistent field within hierarchy. With this hint you can massively copy data from one entities to another. Imagine we have two persistent types Person and Customer. Customer inherits from Person. Customer have a string field ContactPhone. We want to move ContactPhone field to Person. Corresponding hint will look like this:

Using CopyFieldHint:

new CopyFieldHint(
  "Xtensive.Storage.Samples.Upgrade.Model.Customer", "ContactPhone",
   typeof (Person));

We call constructor with three arguments: a name of the source type in the old model, a name of the field being copied, a destination type from the new model. There is optional fourth parameter. It is a name of the field in destination type that will receive the data. If not specified destination field name will be the same as source field name. It's important for source and destination types to have the same structure of key fields, because upgrade code will rely on primary keys to find association between source and destination rows in the database. Source and destination fields should either have the same primitive type, or have complex structure that produces the same set of primitive columns. For example you can easily copy entity reference fields that reference entities with the same Key structure.

Now you know how to create upgrade hints. To add hints you need to override AddUpgradeHints method in your upgrade handler and add required hints to collection. The following code summarizes usage of upgrade hints.

Using upgrade hints:

protected override void AddUpgradeHints()
{
  var hintSet = UpgradeContext.Demand().Hints;
  hintSet.Add(new RenameTypeHint(
    "Xtensive.Storage.Samples.Upgrade.Model.Category", typeof (ProductGroup)));
  hintSet.Add(new RenameFieldHint(
     typeof (Product), "Category", "Group"));
  hintSet.Add(new CopyFieldHint(
     "Xtensive.Storage.Samples.Upgrade.Model.Customer", "ContactPhone", typeof (Person)));
}

Firstly, we demand UpgradeContext and get its Hints property. Then we create hints and add them to the collection.

Using RecycledAttribute

The second way to perform upgrade (so called "manual upgrade") is a manual copying data with aid of RecycledAttribute. This attribute can be applied to either persistenet property or persistent type. It marks its target as unused. Such property/type will not be available during normal work with persistent entities. But it will be available during upgrade stage, so you can use data to manually copy it somewhere. Let's imagine we have a persistent entity with the following structure (some members are omitted for clarity).

Order entity:

[HierarchyRoot]
public class Order : Entity
{
  [Key, Field]
  public long Id { get; private set; }
 
  [Field]
  public Product Product { get; set; }
 
  [Field]
  public int Amount { get; set;}
}

Later we decided that Order won't contain Product and Amount directly, but instead it will posses a set of OrderItem entities each containing Product and Amount.

Refactored Order entity:

public class Order : Entity
{
  [Key, Field]
  public long Id { get; private set; }
 
  [Field, Obsolete, Recycled]
  public Product Product { get; set; }
 
  [Field, Obsolete, Recycled]
  public int Amount { get; set; }
 
  [Field, Association(PairTo = "Order", OnTargetRemove = OnRemoveAction.Cascade)]
  public EntitySet<OrderItem> Items { get; private set; }
}
 
[HierarchyRoot]
public class OrderItem : Entity
{
  [Key, Field]
  public long Id { get; private set; }
 
  [Field]
  public Order Order { get; private set; }
 
  [Field]
  public Product Product { get; set; }
 
  [Field]
  public int Amount { get; set; }
 
  public OrderItem(Order order)
  {
    Order = order;
  }
}

We added [Recycled] attribute to Product and Amount properties. Also we added ObsoleteAttribute to both of them. It is not necessary to mark fields as obsolete but it will produce helpful warnings when these fields are accessed.

Finally, we overriden OnUpgrade method to perform copying data. Both recycled and non-recycled properties are available inside this method. So data copying can be done easily - using your own entities.

Using OnUpgrade method:

public override void OnUpgrade()
{
  foreach (var order in Query<Order>.All)
    order.Items.Add(new OrderItem(order)
    {
      Amount = order.Amount,
      Product = order.Product
    });
}

Example

Complete source code demonstrating both "manual" and "semi-automatic" upgrades can be found in Schema upgrade sample (Xtensive.Storage.Samples.Upgrade) in DataObjects.Net distribution.