Webhook Management

Status

  • Enabled: Webhooks are enabled for this organisation

  • Disabled: Webhooks are not enabled for this organisation

  • Modifying: Webhooks have recently been enabled/disabled

Replication Task Status

  • Running: Webhooks are enabled and processing

  • Stopping: Webhooks have recently been disabled

  • Stopped: Webhooks have been disabled

Replication Task ID

The Replication Task ID relates to the Replication Task within AWS which is created to “migrate” data from the AgencyCloud database to a Kinesis data stream in order to be sent out as a webhook event. In the event of an issue, this ID can be used to debug the issue (relates customer to specific Replication Task).

Replication Task ARN

This is very similar to the above exception instead of the ID, we’re displaying the ARN which could be used to reference the Replication Task elsewhere in AWS.

Replication Instance ARN

A Replication instance is created prior to a Replication Task and this field is displaying the Replication Instance ARN that the Replication Task is associated to. This information is available to aid debugging.

Most Recent Process

Process ID: The Process ID is created by the automation service. This specific ID is the currently running or previously ran ID, it’s used to associate the running tasks and provide updates on the current status, if a change is being made.

Status:

The status on the Replication Task which is provided by AWS:

  • Running

  • Pausing

  • Stopping

  • Stopped

  • Creating

  • Created

  • Ready

  • Modifying

  • Modified

Start: The date the process for enabling/disabled was started

Finished: The date process for enabling/disabling was completed

Workflow ID: The workflow ID that is associated with the Replication Task, this is so that we can run a preconfigured workflow against the Replication Task when enabling/disabling.

Tasks

  • BULK_INDEX_CALL What is it and what does it relate to? This is to start a long running process which obtains all IDs from AC associated to the org and adds the to dyanmodb. This is required to speed up the initial process or skip unrequired changes when these are migrated by the Replication Task so that only recently added rows are synced.

  • AWAIT_BULK_INDEX_FINISH What is it and what does it relate to? As the bulk indexing can take up to 2 hours (or perhaps more) we need to wait for the process to finish before continuing the workflow. Once the bulk indexing is finished, an event will trigger this process task to be completed so that the workflow can continue.

  • PAUSE_REPLICATION What is it and what does it relate to? This is required by AWS as before you can modify a Replication Task first you’re required to pause the Replication Task and because of how it is setup, this task is used to start the pause of the Replication Task

  • AWAIT_REPLICATION_STOP What is it and what does it relate to? This task is awaiting for the Replication to be Paused/Stopped in order to be able to modify the task in the next step, this can take roughly 5 minutes.

  • MODIFY_REPLICATION_AND_RESUME What is it and what does it relate to? This task finally modifies the Replication Task and either resumes the task or sends a notification for it’s scheduled deletion as it’s no longer required

Types

  • COMPUTE: This is a type of task within the automation workflow and will invoke a configured lambda for this particular step

  • EVENT_BRIDGE_TRIGGER: This task is awaiting for an event bridge event to happen to allow the task to be completed to continue the process

  • RESOURCE_CHANGE_EVENT: This task is awaiting for a resource to enter a change event and is waiting for an event bridge event specific for the configured resource (in this case the Replication Task). When the resource changes to a specific state, this task will be executed allowing the process to continue. The next task cannot be completed until this task has finished.

Task types for the automation service are currently documented here: https://github.com/reapit/platform-tools/blob/master/packages/automation-workflow-sdk/README.md#task-types

Status

  • Skipped: The task has been skipped

  • Success/Completed: The task has been successful

  • Failed: The task has failed

  • Running: The task is currently running

  • Queued: The task has been is queued

Enabling Webhooks

  • Click ‘Enable’ from the left-hand navigation

  • Select which Target DMS Endpoint relates to the customer

  • Click ‘Start setup process’

Target DMS Endpoints

  • kinesis-paas-cdc-stream-endpoint: This endpoint should be the default endpoint for most customers

  • kinesis-paas-cdc-stream-endpoint-countrywide: This endpoint should be selected for any customers under the Countrywide/Connells umbrella, who have dedicated parts of the pipeline

  • kinesis-paas-cdc-stream-endpoint-els: This endpoint should be selected for any customers who have just gone live with us, and might be subject to data fixes following a conversion

  • kinesis-paas-cdc-stream-endpoint-projects: This endpoint should be selected for any customers whom we are setting up webhooks for who are in project phase

  • kinesis-paas-cdc-stream-endpoint-savills: This endpoint should be used only for Savills, who have dedicated parts of the pipeline

Depending on the size of the customer and or if bulk indexing was selected it can take a several hours to enable. We are developing a notifications feature but would suggest coming back to the app periodically to check the status.

Disabling Webhooks

  • Click ‘Disable’ from the left-hand navigation

Reenabling Webhooks

  • Click ‘Reenable’ from the left-hand navigation

  • Select which Target DMS Endpoint relates to the customer

  • Select (if applicable) if bulk indexing should be ignored. If so, select ‘Ignore bulk indexing.’ If webhooks have been disabled for a long period of time, it’s recommend to reindex customer data. If they have been disabled for a short period of time (ie to support the running of a bulk script by the service desk, this should be selected to skip the indexing step)

  • Click ‘Continue’ to proceed

FAQ:

What happens if a task fails? Mostly, you’ll be able to enable or disable a second time however this may not always be the cause as this service is dependent on Replication Tasks from AWS

Who should I contact if something isn’t working? Please raise an issue on Github using our bug template here

Why is there no previous process information available for certain customer? This is a new tooling service and collating previous process history is costly and requires extra development work. We can revisit if necessary.

PreviousPlatform Tooling Dashboard

Last updated