Webhook Management
Status
Enabled: Webhooks are enabled for this organisation
Disabled: Webhooks are not enabled for this organisation
Modifying: Webhooks have recently been enabled/disabled
Replication Task Status
Running: Webhooks are enabled and processing
Stopping: Webhooks have recently been disabled
Stopped: Webhooks have been disabled
Replication Task ID
The Replication Task ID relates to the Replication Task within AWS which is created to “migrate” data from the AgencyCloud database to a Kinesis data stream in order to be sent out as a webhook event. In the event of an issue, this ID can be used to debug the issue (relates customer to specific Replication Task).
Replication Task ARN
This is very similar to the above exception instead of the ID, we’re displaying the ARN which could be used to reference the Replication Task elsewhere in AWS.
Replication Instance ARN
A Replication instance is created prior to a Replication Task and this field is displaying the Replication Instance ARN that the Replication Task is associated to. This information is available to aid debugging.
Most Recent Process
Process ID: The Process ID is created by the automation service. This specific ID is the currently running or previously ran ID, it’s used to associate the running tasks and provide updates on the current status, if a change is being made.
Status:
The status on the Replication Task which is provided by AWS:
Running
Pausing
Stopping
Stopped
Creating
Created
Ready
Modifying
Modified
Start: The date the process for enabling/disabled was started
Finished: The date process for enabling/disabling was completed
Workflow ID: The workflow ID that is associated with the Replication Task, this is so that we can run a preconfigured workflow against the Replication Task when enabling/disabling.
Tasks
BULK_INDEX_CALL What is it and what does it relate to? This is to start a long running process which obtains all IDs from AC associated to the org and adds the to dyanmodb. This is required to speed up the initial process or skip unrequired changes when these are migrated by the Replication Task so that only recently added rows are synced.
AWAIT_BULK_INDEX_FINISH What is it and what does it relate to? As the bulk indexing can take up to 2 hours (or perhaps more) we need to wait for the process to finish before continuing the workflow. Once the bulk indexing is finished, an event will trigger this process task to be completed so that the workflow can continue.
PAUSE_REPLICATION What is it and what does it relate to? This is required by AWS as before you can modify a Replication Task first you’re required to pause the Replication Task and because of how it is setup, this task is used to start the pause of the Replication Task
AWAIT_REPLICATION_STOP What is it and what does it relate to? This task is awaiting for the Replication to be Paused/Stopped in order to be able to modify the task in the next step, this can take roughly 5 minutes.
MODIFY_REPLICATION_AND_RESUME What is it and what does it relate to? This task finally modifies the Replication Task and either resumes the task or sends a notification for it’s scheduled deletion as it’s no longer required
Types
COMPUTE: This is a type of task within the automation workflow and will invoke a configured lambda for this particular step
EVENT_BRIDGE_TRIGGER: This task is awaiting for an event bridge event to happen to allow the task to be completed to continue the process
RESOURCE_CHANGE_EVENT: This task is awaiting for a resource to enter a change event and is waiting for an event bridge event specific for the configured resource (in this case the Replication Task). When the resource changes to a specific state, this task will be executed allowing the process to continue. The next task cannot be completed until this task has finished.
Status
Skipped: The task has been skipped
Success/Completed: The task has been successful
Failed: The task has failed
Running: The task is currently running
Queued: The task has been is queued
Enabling Webhooks
Click ‘Enable’ from the left-hand navigation
Select which Target DMS Endpoint relates to the customer
Click ‘Start setup process’
Target DMS Endpoints
kinesis-paas-cdc-stream-endpoint: This endpoint should be the default endpoint for most customers
kinesis-paas-cdc-stream-endpoint-countrywide: This endpoint should be selected for any customers under the Countrywide/Connells umbrella, who have dedicated parts of the pipeline
kinesis-paas-cdc-stream-endpoint-els: This endpoint should be selected for any customers who have just gone live with us, and might be subject to data fixes following a conversion
kinesis-paas-cdc-stream-endpoint-projects: This endpoint should be selected for any customers whom we are setting up webhooks for who are in project phase
kinesis-paas-cdc-stream-endpoint-savills: This endpoint should be used only for Savills, who have dedicated parts of the pipeline
Disabling Webhooks
Click ‘Disable’ from the left-hand navigation
Reenabling Webhooks
Click ‘Reenable’ from the left-hand navigation
Select which Target DMS Endpoint relates to the customer
Select (if applicable) if bulk indexing should be ignored. If so, select ‘Ignore bulk indexing.’ If webhooks have been disabled for a long period of time, it’s recommend to reindex customer data. If they have been disabled for a short period of time (ie to support the running of a bulk script by the service desk, this should be selected to skip the indexing step)
Click ‘Continue’ to proceed
FAQ:
What happens if a task fails? Mostly, you’ll be able to enable or disable a second time however this may not always be the cause as this service is dependent on Replication Tasks from AWS
Who should I contact if something isn’t working? Please raise an issue on Github using our bug template here
Why is there no previous process information available for certain customer? This is a new tooling service and collating previous process history is costly and requires extra development work. We can revisit if necessary.
Last updated