Technical Overview

Introduction#

For a functional overview of how the integration works, see Functional Overview.

The Element451 integration with Ethos is designed to be simple from a user perspective, while handling complex data processing behind the scenes.

Ethos, developed by Ellucian, provides a standardized API that allows Element451 to exchange data with Banner and Colleague. Through Ethos, Element451 can interact with defined fields, properties, and objects within the SIS.

However, Ethos does not expose all data available in Banner or Colleague, and available endpoints may differ between systems.

---

Integration Approach#

Element451 uses a standard integration approach that relies on delivered Ethos functionality whenever possible.

In cases where Ethos does not provide access to required data:

• Banner: Element451 may use the Business Process API (BPAPI)
• Colleague: Partners may need to configure Ethos Extensions for delivered or custom entities

These approaches ensure that required data can still be integrated when it is not available through standard Ethos endpoints.

---

Scope of This Guide#

This section provides a high-level technical overview of what occurs after the integration process is triggered by the webhook (see Functional Overview).

• The process described here reflects the standard integration flow used for most partners
• Some implementations may include customizations or exceptions based on specific requirements

For detailed information on Ethos endpoints, payloads, and data structures, refer to Ellucian’s documentation:

• https://resources.elluciancloud.com/ (opens in a new tab) (Login required to access endpoint documentation)

Primary Tool#

Element451 uses Dagster (opens in a new tab) to orchestrate all integration processes.

Dagster manages the following sync types:

• Element451 → Ethos
• Ethos → Element451
• Data source updates

---

How It Works#

• Dagster runs a series of jobs and sensors to monitor and execute integration activity.
• Within each sync process, multiple operations (ops) are performed to:
  • Extract data
  • Transform data
  • Send or receive data between systems

Each sync is composed of these coordinated steps to ensure data is processed and transmitted correctly across systems.

---

Element451 → Ethos#

Extraction#

When the integration process is triggered, the first step is to load partner-specific configuration settings.

This includes:

• API credentials (Test or Production)
• Ethos configuration values (e.g., name types, email types)
• Element451 taxonomies and IDs
• Field overrides and other configuration settings

These values are retrieved from Element451’s database and AWS Secrets.

Once configuration is loaded, Element451 extracts the contact record using the /v2/users/export API endpoint with a predefined template. This ensures all required data is included for integration with Ethos.

---

Translation#

After extraction, the data is translated into Ethos compatible values.

• Ethos requires GUIDs for most fields rather than SIS codes
• Element451 uses translation tables to map internal IDs and codes to the appropriate Ethos GUIDs

Additional processing includes:

• Translating CEEB codes into Ethos identifiers
• Applying default educational institutions when needed (e.g., GED, Homeschool, Unknown School)

---

Person Matching#

Next, the integration determines how the record should be matched in Ethos.

• If an Ethos ID already exists on the Element451 record, this step is skipped
• If only a School ID (Banner or Colleague ID) exists, Element451 attempts to retrieve the corresponding Ethos ID

If no identifiers are present, Element451 sends a request to: /api/person-matching-request

This request includes identifying data such as:

• Name
• Date of birth
• SSN (if available)
• Emails, addresses, and other identifiers

Ethos applies the institution’s matching rules and returns one of the following:

• newPerson: No match found; a new record is created in Ethos
• existingPerson: A single match is found; the existing Ethos ID is returned
• reviewRequired: Multiple matches found; manual review is required

Once an Ethos ID is identified, Element451 retrieves and stores additional data (if not already present):

• School ID (Banner or Colleague ID)
• School Email (if available)
• A source entry indicating the initial sync date and time

SSN Handling

• Element451 does not overwrite SSNs in the SIS if one already exists
• An optional configuration can also:
  • Compare SSNs between systems
  • Trigger an error if they do not match
  • Apply a label indicating the mismatch

This configuration is disabled by default.

---

Address Updating#

The integration then evaluates and updates addresses using: /api/addresses

• Existing Ethos addresses are compared with Element451 data
• An address is updated only if:
  • The address type matches, and
  • Street Line 1 matches

If no match is found, the address is treated as new and added without modifying existing records in the Person Updating step.

---

Person Updating#

After address processing, the integration updates the person record using: /api/persons

This step ensures the record in Ethos contains complete and up-to-date data, including:

• Names and name types
• Ethnicity and race
• Languages
• Addresses and phone numbers
• Pronouns
• Marital status

This step is required because the person matching process does not include full record data.

Optional Validation

If enabled, the integration will verify:

• Name
• Date of Birth

If these values do not match between systems, the sync will stop and return an error requiring correction before proceeding.

---

Other Operations#

The following operations run after the core person sync steps.

• These operations are independent and may run in any order
• If one operation fails, it does not stop the overall sync
• Errors are isolated to the specific record or operation

---

Create / Update Applications#

All applications associated with the contact are prepared and sent to Ethos using: /api/admission-applications(processed via /api/admission-applications-submissions)

Sync behavior:

• Applications must be:
  • Submitted, and
  • Have a major/program selected
• Applications that do not meet these requirements will:
  • Not be sent, and
  • Generate an error in the Integration Log(optional configuration available to suppress these errors)
• Applications that have moved to student status are not updated:
  • Colleague: Moved to Student (MS)
  • Banner: SGASTDN record exists
• All other applications will be:
  • Created, or
  • Updated in the SIS

Banner Note

• Some Banner environments include programs with concentrations that are not supported by standard Ethos endpoints
• When this condition is detected, the integration uses BPAPI as a fallback

Decision Processing

• Released decisions can be sent after the application (if configured)
• By default, decisions must:
  • Have a valid SIS code, and
  • Be released
• An optional configuration can allow non-released decisions to be sent

---

Create / Update Emergency Contacts#

Emergency contacts are processed using: /api/person-emergency-contacts

Sync behavior:

• If an Ethos ID exists, it is used for matching
• Otherwise, matching is attempted based on name
• If a match is found, by default, it is not updated (as it is presumed the SIS is the system of record). However, an optional configuration can be enabled to allow updating of emergency contacts if needed.
• If no match is found, a new emergency contact is created

Colleague Note

• To designate a contact as a Missing Person contact:
  • Set “Is Primary” = Yes in Element451

---

Create / Update Visa#

Visa records are processed using: /api/person-visas

• If a visa exists on the contact record, it will be created or updated in the SIS

Colleague Note

• This operation creates an FPER record

---

Create / Update Previous Schools#

Previous school records are processed using: /api/person-external-education

Sync behavior:

• Banner: Create and update supported
• Colleague: Create only (no updates)

Colleague Note

• Due to an Ethos limitation, updating existing school records can result in data loss
• Element451 does not update these records in Colleague

CEEB Translation

Element451 maintains a translation table that maps CEEB codes to Ethos IDs.

During sync:

• The CEEB value is matched to an Ethos ID
• The Ethos ID is sent in the payload

If no match is found:

• If default unknown CEEBs are configured, a default value is used
• If not, the sync will return an error

This also applies when a school is entered manually without a CEEB

---

Banner GPA Handling

• The /api/person-external-education endpoint writes GPA to:
  • performanceMeasure
• Some Banner environments do not use this property/field for college GPA

In these cases:

• Element451 can make an additional call to: /api/person-external-education-credentials
• This writes GPA to:
  • SORDEGR_GPA_TRANSFERRED

Note:This endpoint requires a credential/degree to be included for each school.  If a partner wishes to use this endpoint, they will need to notify the Element implementation team.

---

Create / Update Relatives#

Parent and guardian records are processed in two steps:

1. Person record creation or matching
   • Uses /api/person-matching-requests
2. Relationship creation or update
   • Uses /api/personal-relationships

This ensures both the related person and their relationship to the student are properly recorded.

---

Save SSN to Banner (Special)#

• This operation applies to Banner only
• Due to a limitation in older Ethos versions, SSNs may not save correctly using /api/persons (v12)
• To address this, the integration uses an alternative v8 endpoint to store SSNs

---

Custom Operations#

Some implementations include custom operations to support additional integration needs.

• These may interact with:
  • Delivered Ethos endpoints
  • Custom or extended endpoints
• Custom operations are typically executed within this stage of the integration process.

---

Ethos → Element451#

Explanation#

The Ethos → Element451 process uses the Ethos Change Notification (Subscription) system to detect and process updates from the SIS.

Partners are responsible for configuring this system to ensure the Element451 application is subscribed to the appropriate endpoints. At a minimum, the following are recommended:

• addresses
• admission-applications
• person

If change notifications are not being generated and visible within Ethos, Element451 will not be able to detect or process updates.

Important

• Once a notification is consumed from the queue, it cannot be retrieved again
• If the queue is cleared, any unprocessed changes are permanently lost

UAT Behavior

• During UAT, this process is not fully automated
• Notifications are not continuously consumed to avoid excessive data from SIS test environments
• Instead, records are manually pulled from Ethos for testing

---

Processing Overview#

The Ethos → Element451 process is handled through three primary jobs/sensors in the orchestration system:

• Dispatch Ethos Notification
• Pull Ethos Data
• Import Ethos Data to Element

---

Dispatch Ethos Notifications#

• Runs every 5 minutes (in production)
• Retrieves notifications from: /api/consume
• Stores notification data in Element451’s database

Note: This process is typically not exercised during UAT.

---

Pull Ethos Data#

This process is triggered when new notifications are detected.

It retrieves data from multiple Ethos endpoints using the person Ethos ID.

Data Retrieval#

• Person
  • /api/persons
• Addresses
  • /api/addresses
• Applications
  • /api/admission-applications
• Previous Schools
  • /api/person-external-education
  • Additional calls:
    • /api/educational-institutions
    • /api/addresses (institution address)
• Relationships
  • /api/personal-relationships
• Emergency Contacts
  • /api/person-emergency-contacts

---

Data Processing#

• Retrieved data is combined into a single record per student
• The combined record is stored in a staging area in its raw Ethos format

---

Import Ethos Data to Element#

This process is triggered when a complete staging record is available.

It consists of three phases: Extract, Transform, and Import.

---

Extract#

• Retrieve the combined Ethos record from staging
• Retrieve the existing Element451 record (if it exists) using the Ethos ID

---

Transform#

Data is translated into Element451’s format through multiple operations:

• Transform User Profile
  • Prepares core person data (name, gender, email, phone, IDs, etc.)
• Transform User Addresses
  • Converts Ethos address data into Element451 address structure
• Transform User Applications
  • Parses applications into Element451 fields (major, term, campus, student type, etc.)
  • Generates milestones such as:
    • Application Submitted
    • Withdrawn
    • Enrolled (if configured)
• Transform Previous Schools
  • Matches institutions using Ethos ID → CEEB → Element451 school
  • Falls back to name + school type matching if needed

---

Import#

Transformed data is written into Element451 through the following operations:

• Import User Profile
  • Creates or updates the contact record based on configuration settings (this includes their address)
• Import User Applications
  • Creates or updates application records
• Import Milestones
  • Applies milestones generated during transformation
• Import Previous Schools
  • Creates or updates previous school records
• Import Source Codes
  • Adds an “Ethos Latest Sync” source with the current date and time
  • Enables tracking of when data was last synced from Ethos

---

Other Processes#

In addition to record syncing, Element451 maintains data sources and translation tables to support integration with Ethos.

Because Ethos requires GUIDs/IDs for many values, these processes ensure Element451 has the correct data and mappings needed for successful sync operations.

---

Data Sources#

Data sources are collections of values used in Element451 fields (e.g., dropdown options such as majors, terms, or degrees).

• These values are initially built and synced during implementation
• After go-live, the following data sources/translations are refreshed on a weekly schedule:
  • Majors
  • Terms
  • Degrees
• Additional resyncs can be requested through Live Support at any time

Each data source may include additional stored properties, such as:

• Code
• SIS GUID

These properties are used during the build process, but are not directly used during record sync operations.

---

Translation Tables#

Translation tables are internal mappings between Element451 values and Ethos GUIDs.

• These tables are not visible in the UI
• They are used during the translation and transformation steps of the integration process

For each synced data source, a corresponding translation table is created.

---

Data Source Sync Process#

When a data source is synced:

1. Values are retrieved from Ethos
2. Existing values are retrieved from Element451
3. The datasets are merged
4. The updated collection is written back to Element451

---

Special Handling#

Terms#

• Only terms are imported when:
  • No end date is defined, or
  • The end date is a default “end of time” value, or
  • The end date is after the go-live date

---

Majors#

• Majors are imported when:
  • They are active, or
  • Their end date is:
    • A default “end of time” value, or
    • Within the past 10 years

Banner Note

• Ethos may return multiple versions of a program:
  • Program only
  • Program + major/field of study
  • Program + major/field of study + concentration
• Ethos does not indicate which combinations are valid for use
• Partners are responsible for activating the appropriate program combinations in Element451

---

Translation Table Sync Process#

For each synced data source:

• A corresponding translation table is generated using the synced values
• Each entry maps:
  • Element451 value (code or identifier)
  • Ethos GUID/ID

---

Institution Translation Table#

A specialized translation table is created for institutions:

• Retrieves all institutions from Ethos
• Maps: CEEB code → Ethos GUID/ID

During previous school syncing:

• The CEEB is used to identify the institution in Element451
• The translation table is used to retrieve the corresponding Ethos ID

If no match is found:

• A default institution may be used (if configured), or
• The sync will return an error

---

Ethos Endpoints#

The following Ethos API endpoints are used to retrieve data for data sources and translation tables: