Model lifecycle

Once a project has been created, many users might participate in it by building models, comparing their performances, running bulk inferences, creating different bucketings, etc., so these models go through different states as the project makes progress. In order to facilitate managing this process and allow different users to participate in it, the "Model Life Cycle" is introduced. This life cycle represents the different states a model can go through since it is created until it is deployed in a production environment, and/or is eventually deleted. 

The diagram depicted in Figure 1 represents such life cycle, along with the different transitions allowed to happen between states, each of them represented by an arrow. Some things to consider about the model life cycle:

• Please note that some transitions are bidirectional, while others are not. 
• Some of these transitions can happen automatically, performed by the "housekeeping" process, after a predefined period of time. These periods of time can be defined by Admins at an organisation level.
• Manual transitions can be performed from the "Performance" screen, where the model summary is shown (Figure 2). The drop-down list will only contain the allowed transitions at every state.
• All transitions can also be performed from the "Model Manager User Interface", detailed in this section.
• Every time a transition happens, a text box will pop up asking the user to input an explanation for the transition. These inputs are stored in the database for auditing purposes, allowing the tracking of which user made which specific action, and the reasons to do so.

Figure 1: Diagram of the Model Life Cycle, along with the allowed transitions between states.

Each state within the Model Life Cycle is described in the following:

• Built: the initial state after a model has been created. 
  • NOTE: All models which have been in the "Built" state for a predefined period of time without any changes will automatically be moved to the "Trash bin". The reversed action (from "Trash bin" to "Built") cannot happen automatically.
• Ready for analysis: built models whose performance is valid/acceptable can be labelled as "ready for analysis". This allows model builders to label good candidates, and stakeholders/decision-makers to easily spot these candidates to be deployed in a later phase, before accepting/rejecting them.
• Accepted: a model "ready for analysis" can be accepted before it is deployed.
• Rejected: a model "ready for analysis" can be rejected by a decision-maker.
  • NOTE: All models which have been in the "Rejected" state for a predefined period of time without any changes will automatically be moved to the "Trash bin".
• Deployed: models enable and expose an API when they get to the "Deployed" state. At this stage (not before) they can respond to API calls, including the ones performed from the "Inference" page. 
• Live: in order to perform integration tests and run live inferences, models must be deployed; however, not all deployed models are later integrated into a production environment. The "Live" state allows the user to flag those models which are integrated into production. However, this is just an external label to distinguish models and ease their management. Internally, the "Deployed" and "Live" states are indistinguishable. 
• Undeployed: models can transition to this state only from "Deployed/Live"; this transition shuts down the API, which becomes unavailable. 
  • NOTE: All models which have been in the "Undeployed" state for a predefined period of time without any changes will automatically be moved to the "Trash bin".
• Archived: this state is intended to represent models that were deployed/live in the past but are no longer used. This allows the organisation to keep an archive of all models that were in production in the past, without losing the history of inferences and other associated data/information.
• Trash bin: this state describes all models that have been discarded but can still be recovered. In order to recover a model, a manual transition to the "Built" state must be performed. 
  • NOTE: All models which have been in the "Trash bin" for a predefined period of time without any changes will be automatically deleted.
• Purged: purged models are erased from the database and are not recoverable.

Figure 2: manual transitions can be performed from the "Performance" screen, where the model summary is shown.