JavaScript Ramblings

Sync mirrored GitLab repository automatically on every push to GitHub

Corina -

Did you successfully setup pull mirroring between GitHub and GitLab, but are disappointed to see changes only get synced every 30 minutes?

Luckily, GitLab does support automatically pulling on every push to GitHub, it's just a bit hidden in their docs 😅

In my case, I setup Gitlab as CI/CD for a GitHub external repository. Everything worked well when syncing by hand, but whenever a new pull request was opened in GitHub, the CI job was not triggered immediately. This was of course a big issue, as no one wants to wait for 30min for their build to be triggered 😀

To get immediate sync enabled, we just need to configure a webhook in GitHub to notify GitLab of every new commit.

#1 - Prerequisites

First, let's make sure the basic setup is in place:

  • In GitLab >> Settings >> Repository, you configured mirror pulling from your GitHub repository
  • You can manually trigger a sync and it is successful
  • You enabled GitHub integration under GitLab >> Settings >> Integrations

#2 - Generate a personal access token in GitLab

This is to allow GitHub access to the GitLab project, to notify it of new commits.

I recommend creating a special GitLab "bot" user for these kind of operations.

Go to GitLab >> User >> Settings >> Access tokens and create a new personal access token with api scope.

#3 - Setup a webhook in GitHub to ping GitLab on every change

  • In your GitHub repository, go to Settings >> Webhooks >> Add webhook
  • Under "Payload URL" enter https://<GITLAB_INSTANCE>/api/v4/projects/<PROJECT_ID>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN>
  • > GITLAB_INSTANCE is either gitlab.com or the URL of your on-premise setup
  • > PROJECT_ID is the URL encoded path to your project - if your repo is at my_username/my_project then your PROJECT_ID will be my_username%2Fmy_project; alternatively, you can use the actual numeric id of the project
  • > PERSONAL_ACCESS_TOKEN is a Personal Access Token you need to generate in GitLab to allow GitHub to connect; it only requires Api privileges
  • Hit save!

#4 - Check that everything works

Open a pull request or push a new commit to GitHub - updates should be visible immediately!
The best way to see that everything worked is to go to GitHub >> Webhooks and click the webhook you just created. GitHub shows a list of all events that triggered the hook to be called and whether they were successful or not. This is also great for further troubleshooting issues, in case this doesn't work from the first try.

That's it!

I hope you mananged to get the setup to work from the first try 🤞

References