别不信，5种Gitlab数据迁移方法，让你一劳永逸？

神州数码云基地 2022-08-24

2614

TiDB

神州数码云基地

在 TiDB 上的尝试、调研与分享

本期内容

Gitlab之间的数据迁移

数据迁移；数据互传；数据同步，一直是很多项目中的难题。问题虽小，但作用却大！总有人在寻找处理数据的方法中花费了太多时间...

本期内容我们分享5种Gitlab之间数据迁移的方式，各有优劣，供大家参考，祝找到适合自己的方法~

建议先放至收藏夹，以免丢失！！

①

Gitlab project

export import

任意一个Gitlab实例中的项目，都能通过导出成文件然后导入到新Gitlab的方式来完成迁移。

优点

支持Gitlab之间的数据迁移，故而可以从多个Gitlab迁移数据到一个Gitlab；
操作简单，流程不是很复杂，可以通过调用API实现导入导出。

缺点

不支持增量数据同步；
受到 rate limit的影响，速率相对较慢；
有版本限制，不同版本之间并不通用。

关于限制

# 版本限制

Gitlab 的v13.0之前，只能在兼容版本的Gitlab之间实现导入导出。

例如Gitlab v12.10导出的项目只能导入到版本号为v11.70到v12.10的Gitlab，具体参考：

从GitLab 13.0开始，GitLab可以导入从不同的GitLab部署中导出的包，仅限于之前的两个GitLab小版本，例如：

# 速率限制

# 导出限制

在项目导出时，不是所有的数据都会被导出，有一部分数据并不会导出，具体参考：

The following items are exported:

Project and wiki repositories
Project uploads
Project configuration, excluding
integrations
Issues with comments, merge requests with diffs and comments, labels, milestones, snippets, time tracking, and other project entities
Design Management files and data
LFS objects
Issue boards
Pipelines history
Push Rules
Awards
Group members are exported as project members, as long as the user has the Maintainer role in the exported project’s group, or is an administrator

The following items are not exported:

Child pipeline history
Build traces and artifacts
Package and container registry images
CI/CD variables
Pipeline triggers
Webhooks
Any encrypted tokens
Merge Request Approvers and the number of required approvals
Repository size limits
Deploy keys allowed to push to protected branches
Secure Files

These content rules also apply to creating projects from templates on the group or instance levels, because the same export and import mechanisms are used.

导入导出过程

# 导出

On the top bar, select Menu >
Projects and find your project.
On the left sidebar, select Settings >
General.
Expand Advanced.
Select Export project.
After the export is generated, you should receive an email with a link to download the file.
Alternatively, you can come back to the project settings and download the file from there or generate a new export. After the file is available, the page should show the Download export
button.

# 导入

When creating a new project, select Import project.
In Import project from, select GitLab export.
Enter your project name and URL. Then select the file you exported previously.
Select Import project to begin
importing. Your newly imported project page appears shortly.

②

Gitlab backup

& restore

根据官方文档说，单个gitlab之间数据的迁移，这是最推荐的方式~

优点

流程简单，依赖于Gitlab自带工具；
从Gitlab v14.9 开始支持增量数据备份与恢复;
可以根据需要，选择备份与恢复的内容。

缺点

不支持多个Gitlab到一个Gitlab的数据同步；
恢复时可能会产生数据覆盖，也就是恢复的Gitlab最好是一个空的Gitlab；
同样受限于版本，只支持相同版本Gitlab之间的backup & restore ，You can only restore a backup to exactly the same version and type (CE/EE) of GitLab that you created it on；
不能直接识别使用对象存储保存的项目，需要工具转换为文件存储；

Backup

GitLab provides a command line interface to back up your entire instance, including:

Database
Attachments
Git repositories data
CI/CD job output logs
CI/CD job artifacts
LFS objects
Terraform states (introduced in GitLab 14.7)
Container Registry images
GitLab Pages content
Packages (introduced in GitLab 14.7)
Snippets
Group wikis

Backups do not include:

Mattermost data
Redis (and thus Sidekiq jobs)

# 命令

# Install gitlab by using the Omnibus package
## Gitlab 12.2 or later
sudo gitlab-backup create
## Gitlab 12.1 and earlier
sudo gitlab-rake gitlab:backup:create




# Install gitlab from source
cd home/git/gitlab
sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
复制

# 额外需要备份文件

The backup Rake task GitLab provides does not store your configuration files. The primary reason for this is that your database contains items including encrypted information for two-factor authentication and the CI/CD secure variables. Storing encrypted information in the same location as its key defeats the purpose of using encryption in the first place.

For Omnibus:

/etc/gitlab/gitlab-secrets.json
/etc/gitlab/gitlab.rb

For installation from source:

/home/git/gitlab/config/secrets.yml
/home/git/gitlab/config/gitlab.yml

# Backup options

具体参考：

The command line tool GitLab provides to backup your instance can accept more options.

backup strategy option
STRATEGY=copy

## For Omnibus GitLab packages
sudo gitlab-backup create STRATEGY=copy


## For installations from source
sudo -u git -H bundle exec rake gitlab:backup:create STRATEGY=copy RAILS_ENV=production
复制

# skip db and uploads
## For Omnibus GitLab packages
sudo gitlab-backup create SKIP=db,uploads




## For installations from source
sudo -u git -H bundle exec rake gitlab:backup:create SKIP=db,uploads RAILS_ENV=production
复制

Restore

# Prerequisites

# Command

# Restore for installation from source
## Stop processes that are connected to the database
sudo service gitlab stop
## restore
sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production
复制

参考：

Back up and restore GitLab | GitLab

③

Git clone、pull

& push

通过git的方式来实现数据的迁移，是一种非常简单且机械的方式，不用考虑多到单，单到单、全量迁移、增量迁移等种种问题。

通过git clone Gitlab-A / Gitlab-B 等到本地，然后git push到Gitlab-new。

有增量的数据，可以pull拉取最新的代码，然后push到新Gitlab，以此来实现增量的更新。

缺点

需要在迁移的Gitlab中提前建好git库，这样才能push到新Gitlab；（不过可以通过写代码，获取API token，调用API来做）
只能迁移git中存储的信息，那些存储在gitlab中的并不能实现迁移，比如merge记录、CI/CD记录等等信息。

④

Gitlab import

bare repostries

Rake tasks are available to import bare repositories into a GitLab instance.

When migrating from an existing GitLab instance, and to preserve ownership by users and their namespaces, please use our project-based import/export. （也就是使用方式一）

注意事项

The owner of the project is the first administrator.
The groups are created as needed, including subgroups.
The owner of the group is the first administrator.
Existing projects are skipped.
Projects in hashed storage may be skipped. For more information, see Importing bare repositories from hashed storage. 从Gitlab 10.0开始引入hashed storage，12.0之后作为默认存储方式，不过迁移方式类似
The existing Git repositories ware moved from disk (removed from the original path).

⑤

Gitlab官方

Congregate 工具

Congregate is a migration tool that can move data from many different Source Code Management (SCM) and Continuous Integration (CI) systems into a GitLab Self Managed (SM) instance hosted in the customer data center/cloud or GitLab.com.

It is what GitLab Professional Services uses for large scale migration engagements.

It is being shared in a source-available way for collaboration with Customers and Partners.