|
The Data Object Model (DOM) is a set of classes that is used to handle scenario data in memory when writing complex processing. DOC provides internal mechanisms to map this data model to the Data Service relational database and most extension points related to data processing will receive the data they need in the form of a ready-to-use generated class called Collector. The DOM is available for the Java and the Python programming language.
A collector provides the following functions:
Creation and deletion of an instance of a given entity,
Access to all instances of a given entity,
Access to a specific instance using its business key, and
Read/Save its content into/from a snapshot text file.
DOM classes are generated from the JDL Business Data Model description and can be found in the gene-model-dom and gene-model-dom-python libraries of the generated application projects.
DOC assumes that all scenario data processing code uses a collector as input and provides a collector as output.
Typical examples include:
Model Checking
A collector containing the scenario data to be checked is received as input of the checker.
The checker code uses the collector API to browse through the different instances and to detect potential issues.
When problems are detected, the checker creates GeneIssue instances in the provided collector.
The collector in then saved back into the scenario data.
Optimization Engine Computation
A collector containing the scenario data to be processed is received as input of the engine.
The engine code uses the collector API to set up the optimization engine constraints and objectives.
The solution is extracted from the engine and appropriate instances are created in the provided collector to store the solution.
The collector in then saved back into the scenario data.
This principle allows to hide the scenario data mapping complexity and enables developing the processing features without concerns on the future context of execution:
Unit tests rely on collectors manually created from snapshot files.
Deployed code rely on collectors automatically created by DOC in the backend/worker task contexts.
The DOM is generated during the build phase of the gene-model-dom and gene-model-dom-python projects. Whether it is or not a Composite Data Model application, the generation is performed using the JDL files located in gene-model/spec. These file contains the entities and relationships that define the application data model. For more details, refer to Chapter Defining the Data Model and Section Capacity Planning JDL Samples.
The generated code can be found in the gene-model/gene-model-dom/build/generated/src and gene-model/gene-model-dom-python/python/src folders. As the naming implies, those files should NOT be modified since they will be overwritten any time a regeneration is performed.
The generated code includes:
A collector class (whose name is computed from the entities.jdl application java.collectorClass property).
A factory class used to create collectors.
A class for each of the entities defined in the entities.jdl file.
During the build phase, two files are generated for each entity: one interface file and one implementation file.
Here is an example of a set of generated interface files:
1155 Nov 5 18:16 Activity.java 3244 Nov 5 18:16 CapacityPlanning.java 132 Nov 5 18:16 CapacityPlanningFactory.java 1019 Nov 5 18:16 GeneIssue.java 897 Nov 5 18:16 GeneParameter.java 662 Nov 5 18:16 Precedence.java 892 Nov 5 18:16 Requirement.java 985 Nov 5 18:16 Resource.java 779 Nov 5 18:16 ResourceCapacity.java 1001 Nov 5 18:16 ResourceUsagePerDay.java 878 Nov 5 18:16 Schedule.java 868 Nov 5 18:16 SolutionSummary.java
During the build phase, a Python module that contains a class for each entity is generated.
Here is an example of a set of generated entities:
class Activity(DbDomObject, IndexKey): ... class CapacityPlanning: ... class CapacityPlanningFactory: ... class GeneIssue: ... class GeneParameter: ... class Precedence: ... class Requirement: ... class Resource: ... class ResourceCapacity: ... class ResourceUsagePerDa: ... class Schedule: ... class SolutionSummary: ...
Collectors are usually created and populated with data by DOC and thus do not need to be created explicitly except when working on unit tests.
In Java a collector is created using the generated factory class.
CapacityPlanningFactory factory = new CapacityPlanningFactoryImpl(); CapacityPlanning coll = factory.createCollector();
The collector class is an implementation of DbDomCollector.
In Python a collector is created directly using the constructor.
coll = CapacityPlanning()
Instances should not be created by calling the constructor. The collector class includes a set of createXXX methods that should be used instead. Each creation method can be used to create an instance of the corresponding XXX entity.
The following code shows an example of createXXX methods generated to cover the set of entities defined in the tutorial:
public interface CapacityPlanning extends DbDomCollector {
Activity createActivity();
GeneIssue createGeneIssue();
GeneParameter createGeneParameter();
Precedence createPrecedence();
Requirement createRequirement();
Resource createResource();
ResourceCapacity createResourceCapacity();
ResourceUsagePerDay createResourceUsagePerDay();
Schedule createSchedule();
SolutionSummary createSolutionSummary();
...
}To create a resource, one would declare the following:
Resource res1 = coll.createResource();
Each instance is linked with its collector which can be accessed using the getCollector() method.
res1.getCollector() == coll; // TRUE
The following code shows an example of createXXX methods generated to cover the set of entities defined in the tutorial:
class CapacityPlanning(DbDomCollector):
def create_activity(self) -> Acticity:
pass
def create_gene_issue(self) -> GeneIssue:
pass
def create_gene_parameter(self) -> GeneParameter:
pass
def create_precedence(self) -> Precedence:
pass
def create_requirement(self) -> Requirement:
pass
def create_resource(self) -> Resource:
pass
def create_resource_capacity(self) -> ResourceCapacity:
pass
def create_resource_usage_per_day(self) -> ResourceUsagePerDay:
pass
def create_schedule(self) -> Schedule:
pass
def create_solution_summary(self) -> SolutionSummary:
passTo create a resource, one would declare the following:
Resource res1 = coll.createResource();
Each instance is linked with its collector which can be accessed using the getCollector() method.
res1.getCollector() == coll; // TRUE
The generated classes provide an API to get and set the different attributes and relationships of each instance.
We will use the following JDL entity declarations to illustrate the different ways to access and modify instances in Java and in Python:
entity Resource {
// DOM [primary.keys] : [id]
id String required,
name String
}
entity ResourceCapacity {
quantity Integer
}
relationship OneToOne {
// DOM [affects.primary.key] : [true]
ResourceCapacity{resource} to Resource{capacity}
}In Java, the JDL declaration translates in the following Resource and ResourceCapacity interfaces:
public interface Resource extends DbDomObject {
ResourceCapacity getCapacity();
CapacityPlanning getCollector();
String getId();
String getName();
void setCapacity(ResourceCapacity obj);
void setId(String value);
void setName(String value);
}
public interface ResourceCapacity extends DbDomObject {
CapacityPlanning getCollector();
int getQuantity();
Resource getResource();
boolean isSetQuantity();
void setQuantity(int value);
void setResource(Resource obj);
void unsetQuantity();
}For example, to set the attributes of an instance, one would call
Resource res1 = coll.createResource(); res1.setId( "MyFirstResourceId" ); res1.setName( "MyFirstResourceName" );
Relationships are managed in a similar way:
ResourceCapacity capa1 = coll.createResourceCapacity(); capa1.setQuantity(1); capa1.setResource(res1); // this automatically calls Resource.setResourceCapacity() res1.getResourceCapacity() == capa1; // true ResourceCapacity capa2 = coll.createResourceCapacity(); capa2.setQuantity(2); res1.setResourceCapacity(capa2); // this automatically update the inverted relationships res1.getResourceCapacity() == capa2; // true capa1.getResource() == null; // true capa2.getResource() == res1; // true
In Python, the JDL declaration translates in the following Resource and ResourceCapacity interfaces:
class Resource(DbDomObject):
def get_capacity(self) -> ResourceCapacity:
pass
def get_id(self) -> str:
pass
def get_name(self) -> str:
pass
def set_capacity(self, capacity: ResourceCapacity | None):
pass
def set_id(self, id_: str):
pass
def set_name(self, name: str | None):
pass
class ResourceCapacity(DbDomObject, IndexKey):
def get_quantity(self) -> int:
pass
def get_resource(self) -> Resource:
pass
def set_quantity(self, quantity: int | None):
pass
def set_resource(self, resource: Resource | None):
passFor example, to set the attributes of an instance, one would call
res1 = coll.create_resource() res1.set_id( "MyFirstResourceId" ) res1.set_name( "MyFirstResourceName" )
Relationships are managed in a similar way:
capa1 = coll.create_resource_capacity() capa1.set_quantity(1) capa1.set_resource(res1) # this automatically calls Resource.set_resource_capacity() res1.get_resource_capacity() == capa1 # true capa2 = coll.create_resource_capacity() capa2.set_quantity(2) res1.set_resource_capacity(capa2) # this automatically update the inverted relationships res1.get_resource_capacity() == capa2 # true capa1.get_resource() == None # true capa2.get_resource() == res1 # true
When primary key information is provided in the JDL declaration, it can be used to access individual instances from their collector using a generated API.
The following API is available in the Java collector class for example:
public interface CapacityPlanning extends DbDomCollector {
Resource getFromResource(String id);
ResourceCapacity getFromResourceCapacity(String resourceId);
...
}In Python, the collector class provides the following API:
class CapacityPlanning(DbDomCollector):
def find_resource_by_business_key(self, id: str) -> Resource | None:
pass
def find_resource_capacity_by_business_key(self, id: str) -> ResourceCapacity | None:
passHere are some examples of use of the API in Java:
Resource res1 = coll.createResource(); // res1 is added to the collector instances 1 == coll.getResource().size(); // is true res1.setId( "MyFirstResourceId" ); // res1 primary key is complete so can be accessed with getFromResource res1 == coll.getFromResource( "MyFirstResourceId" ); // is true
Instances can also be accessed through the list of all known instances.
public interface CapacityPlanning extends DbDomCollector {
List<Resource> getResource();
List<ResourceCapacity> getResourceCapacity();
...
}Entities that are declared as single.row are accessed as singletons: the following JDL declaration
entity SolutionSummary {
// DOM [single.row] : [true]
start Instant,
end Instant,
timespan Integer
}Will cause the following API to be generated:
public interface CapacityPlanning extends DbDomCollector {
SolutionSummary getSolutionSummary();
void setSolutionSummary(SolutionSummary value);
...
}Here are some examples of use of the API in Python:
res1 = coll.create_resource() # res1 is added to the collector instances 1 == len(coll.get_all_resource()) # is True res1.set_id( "MyFirstResourceId" ) # res1 primary key is complete so can be accessed with find_resource_by_business_key res1 == coll.find_resource_by_business_key( "MyFirstResourceId" ) # is True
Instances can also be accessed through the list of all known instances.
class CapacityPlanning(DbDomCollector):
def get_all_resource(self) -> FrozenSet[Resource]:
pass
def get_all_resource_capacity(self) -> FrozenSet[ResourceCapacity]:
passEntities that are declared as single.row are accessed as singletons: the following JDL declaration
entity SolutionSummary {
// DOM [single.row] : [true]
start Instant,
end Instant,
timespan Integer
}Will cause the following API to be generated:
class CapacityPlanning(DbDomCollector):
def find_solution_summary(self) -> SolutionSummary | None:
pass
def set_solution_summary(self, solution_summary: SolutionSummary) -> SolutionSummary | None:
passTo remove an instance from a collector in Java, one should call:
coll.getResource().remove(res1); 0 == coll.getResource().size(); // is true null == coll.getFromResource( "MyFirstResourceId" ); // is true res1.getResourceCapacity() != null; // may be true. Removing an instance does not clear its relationships.
The following code in Python achieves the same goal:
coll.remove(res1) 0 == len(coll.get_all_resource()) # is True None == coll.find_resource_by_business_key( "MyFirstResourceId" ) # is True res1.get_capacity() == None # True: In python removing an entity from the collector clean its relationships
Note that:
|
A collector content can be saved in and loaded from multiple formats.
zipped csv, which is a set of CSV files put into a zip archive. This format is the default one.
dbrf, which is a CSV based format
xlsx, which is the Excel format.
Only the Excel format can be edited manually.
To save or load a collector, you should call the following functions:
File snapshot = new File("xxx"); // File format will be deducted from the file extension (.xlsx, .zip, .dbrf, .dbrf.gz, .xcsv).
coll.saveSnapshot(snapshot);
coll.loadSnapshot(snapshot);In Python the xcsv is the only format supported. One can call the following functions to save or load a collector:
snapshot_file = "xxx" # When manipulating a collector structure coll = load_collector(snapshot_file, entity_filter) save_collector(coll, snapshot_file, entity_filter) # When manipulating dataframe structures dataframe_dict = load_data_frame_dict(snapshot_file, entity_filter) save_data_frame_dict(dataframe_dict, snapshot_file, entity_filter)
Note that snapshots are tightly linked with the business data model. They include references to class names and packages. entity_filter is optional and indicates which entities to load. If not provided, all entities are loaded. |
In some specific cases, the generated DOM Java classes require to be customized, for instance when migrating a legacy DOC 3.x application.
Note that DOM Python classes are not customizable. This section only applies to Java. |
The first step is to modify the code generation configuration in the gene-model/gene-model-dom/build.gradle file:
modelGeneration {
jdlFile = parent.file("spec/entities.jdl")
javaPackage = "com.example.caplan"
useCustomCode = true
}
sourceSets.main.java.srcDir "src/dom/java"Above:
useCustomCode specifies that the generated classes may contain custom code. When set to true, the code generator will keep the custom code of the DOM classes when classes are re-generated.
sourceSets.main.java.srcDir must be modified to use the new generated code directory src/dom/java by default instead of ${buildDir}/generated/src/main/java.
The project is now configured to allow customization of the generated code.
Each element of the generated code uses a specific annotation in order to distinguish generated code from custom code.
The custom code can be added at the bottom of the class, after the generated code.
For example, custom imports need to be added outside the block <generated-imports>.
// <generated-imports>
...
// </generated-imports>
import java.util.Map;
@Generated(value = "DomGenerator")
public interface Plant extends DbDomObject {
@GeneratedMethod
List<Activity> getActivities();
...
/**
* @return the sum of the activities duration that are using this plant
*/
Double getDurationInHours();
}// <generated-imports>
...
// </generated-imports>
import java.util.Map;
@Generated("DomGenerator")
public class PlantImpl extends DbDomObjectImpl implements Plant {
@GeneratedField
protected List<Activity> activities;
...
@Override
@GeneratedMethod
public List<Activity> getActivities() {
return this.activities;
}
...
/**
* @return the sum of the activities duration that are using this plant
*/
@Override
public Double getDurationInHours() {
return getActivities().stream().mapToDouble(Activity::getDurationInHours).sum();
}
}