Element451Integrations Guide

Technical Overview

Updated Article

Introduction#

For a functional overview of how the integration works, see Functional Overview.

The Element451 integration with Populi is designed to support bi-directional data movement between Element451 and Populi while keeping the user-facing experience simple. Behind the scenes, the integration uses configurable jobs, sensors, translation logic, and client-specific settings to determine how records are extracted, transformed, matched, and written between systems.

The Populi integration library supports multiple workflow types, including:

  • Element451 → Populi
  • Populi → Element451

Scope of This Guide#

This section provides a high-level technical overview of what occurs after a Populi integration process is triggered.

The process described here reflects the standard Populi integration flow used for most partners. Some implementations may include customizations or exceptions based on the partner’s requirements, configuration, or statement of work.

For detailed information on Populi’s endpoints, payloads, and data structure, refer to Populi’s API documentation: https://support.populiweb.com/hc/en-us

Primary Tool#

Element451 uses Dagster to orchestrate Populi integration processes.

Dagster manages the following Populi sync types:

  • Element451 → Populi
  • Populi → Element451

How It Works#

  • Dagster runs a series of jobs and sensors to monitor and execute integration activity.
  • Within each sync process, multiple operations (ops) are performed to:
    • Extract data
    • Transform data
    • Send or receive data between systems

Each sync is composed of these coordinated steps to ensure data is processed and transmitted correctly across systems.

Element451 → Populi#

Extraction#

The first step in the process is to load partner-specific configuration settings and export the student from Element.

This includes:

  • API credentials
  • Element451 taxonomies and IDs
  • Other configuration settings

These values are retrieved from Element451’s database and AWS Secrets.

Once configuration is loaded, Element451 extracts the contact record using the /v2/users/export API endpoint with a predefined template. This ensures all required data is included for integration with Populi.

Translation#

After the student is exported from Element, the data is translated into Populi compatible values.

This includes values such as:

  • Academic Load
  • Terms
  • Campuses
  • Degrees
  • Majors
  • etc

Person Matching / Creation#

Once translation is complete, the integration attempts to find or create the matching person in Populi.

This is done using the following:

  • Element Identity “School ID”
  • Or a combination of first name, last name, date of birth, gender, and preferred name.

If no match is found, a new student is created in Populi. Once a match or creation occurs, the Populi ID is saved back into Element in the School ID identity.

Update Student#

After the Populi person is identified, the process populates the full student data in Populi.

The data that is set/updated in Populi includes:

  • Name
  • Preferred Name
  • Former Name
  • Gender
  • Birth date
  • SSN
  • Address
  • Ethnicity
  • Citizenship Country
  • Phone Numbers
  • Email Addresses

Create / Update Applications#

Next, the applications is created or updated in Populi for the student.

The standard Element → Populi field list includes application data such as:

  • major
  • degree
  • campus
  • term
  • started date
  • submitted date
  • lead source
  • status
  • high school graduation date
  • CEEB code

Create / Update Previous Education#

Previous education records (schools) are synced into Populi.  They are loaded in as organizations.

Create / Update Custom Fields#

The next process is to load any custom fields that have been identified by the partner.  These can vary by partner and where the data is loaded.

Student Role, Student Info, Notes, Program, and Degree#

After core student, application, organization, and custom-field data is handled, the final piece of the integration process is to update specific pieces in Populi:

  • Make sure the record in Populi has the student role associated with them.
  • Set the student’s campus in Populi
  • Add notes from Element into Populi’s notes.
  • Set the student’s program and degree

These steps attach configured student-role and academic details to the Populi record.

Populi → Element451#

Explanation#

The Populi → Element process happens in two stages:

  1. Populi → Staging Area in Database
  2. Staging Area in Database → Element451

The first stage happens on a schedule of hourly, while the second stage is auto triggered by completion of the first stage on a record.

Populi → Staging Area#

This process fetches change notifications from Populi and saves them to the staging area in our database. For each notification, it retrieves the full student record from Populi and saves that extended record.

This is done by retrieving change notifications through the Populi persons route, filtering for records where info_changed is greater than a provided date, saving notifications internally and then saving extended student data to our staging table.

Staging Area → Element#

This process is triggered when a complete staging record is available.

It consists of three phases: Extract, Transform, and Import.

Extract#

  • Retrieve the Populi record from staging

Transform#

Data is translated into Element451’s format values.  This consists of converting Populi values for lookups (like Race and Gender) and translating them into the correct Element data source values.

Additionally, the entire raw Populi record is formatted for Element’s import payload design.

Import#

The transformed data from Populi is then imported using Element’s API (/v2/users/import route).

Aside from the data, a source “Populi Latest Sync” is added to Element’s Source codes collection with the current date/time indicating the record was sync’d in.