github rss
Set Up Gitea to Mirror Your GitHub Organization
Nov 29, 2017

If you’re like me, and are a bit hesitant to use centralized services, in fear, that one they will go away, or start to have degraded service. Well I am a computer programmer (among other things), and that means one of the main services I use is GitHub. GitHub has a great community of open source projects, and is the leader in hosted Git. They have had many visible outages, and so I’ve been looking at ways of setting up a mirror for my git repositories so that if GitHub goes down, then I can still continue to develop.

The first thing that I had to do was to chose a software solution that can host my mirror. Options include GitLab, Gogs, and Gitea. I’ve used all three in the past, but found that GitLab was resource intensive. Also I’ve found that Gogs was slow to update. So with that in mind I chose Gitea (which is a fork of Gogs with a very active community which I am a part of). Gitea is very lightweight and so I can run it on my Raspberry Pi.

This approach will also work for migrating all your git repos as well, instead of just mirroring (in case you plan on leaving GitHub entirely).

Assuming you have Gitea setup, you’ll need to make an API token that can be used to call the Gitea API. You can create a token for yourself on the following page: https://gitea.example.com/user/settings/applications (note: you’ll need to point at your gitea instance). Next, you’ll need to get a similar token from GitHub, which you can create one for yourself here: https://github.com/settings/tokens (note: If the code doesn’t work, then you’ll need to provide more permissions to the token).

Now that you have your access tokens for both Gitea, and GitHub, you’ll need to make sure you have a working installing of Go https://golang.org/ as it is what I used to create this program. Next you’ll need to get go, to run go get on the following libraries:

"code.gitea.io/sdk/gitea"
"github.com/google/go-github/github"
"golang.org/x/oauth2"

Add those libraries, along with the built in ones of context, and time as imports to the start of a go file.

The next step is to create a main function in go that will wrap all the below code.

The first part of the main function will be to create a connection along with authentication to GitHub

ctx := context.Background()
ts := oauth2.StaticTokenSource(
	&oauth2.Token{AccessToken: "GETTOKENFROMGITHUB"}, // TODO: GET TOKEN FROM GITHUB
)
tc := oauth2.NewClient(ctx, ts)

And the same for Gitea

client := github.NewClient(tc)
giteaclient := gitea.NewClient("https://YOURGITEAHOST/", "GETTOKENFROMGITEA")
// TODO: CONFIGURE WITH URL of your GITEA INSTANCE AND A TOKEN FROM YOUR GITEA INSTANCE

Now we need all of the GitHub repos to mirror (this mirrors a GitHub organization for all of their public repositories, but you can change this to mirror the public repositories of a user by looking at the docs for the GitHub golang library).

opt := &github.RepositoryListByOrgOptions{
	ListOptions: github.ListOptions{Type: "public", PerPage: 100},
}

// get all pages of results
var allRepos []*github.Repository
for {
	repos, resp, err := client.Repositories.ListByOrg(ctx, "PUTGITHUBORGYOUWANTCLONEDHERE", opt)
 // TODO: SET WITH NAME OF GITHUB ORG YOU WANT CLONED
	if err != nil {
		return err
	}
	allRepos = append(allRepos, repos...)
	if resp.NextPage == 0 {
		break
	}
	opt.Page = resp.NextPage
}

With all of the repos now stored in a list, you can loop on them and mirror (or migrate them) on Gitea. Please note the few things you’ll need to change in the config below:

for i := 0; i < len(allRepos); i++ {
		description := ""
		if allRepos[i].Description != nil { // will throw a nil pointer error if description is passed directly to the below struct
			description = *allRepos[i].Description
		}
		giteaclient.MigrateRepo(gitea.MigrateRepoOption{
			CloneAddr:   *allRepos[i].CloneURL,
			UID:         4, // TODO: SET WITH THE ID OF YOUR USER IN GITEA (IN MY CASE 4 is the user id of an org on my gitea instance)
			RepoName:    *allRepos[i].Name,
			// Mirror:      true, // TODO: uncomment this if you want gitea to periodically check for changes
			// Private:     true, // TODO: uncomment this if you want the repo to be private on gitea
			Description: description,
		})
		time.Sleep(100 * time.Millisecond) // THIS IS HERE SO THE GITEA SERVER DOESNT GET HAMMERED WITH REQUESTS
}

Now when you put the above code all together and run it, you’re repositories will migrate to your Gitea setup. If the service you are trying to mirror, or migrate from doesn’t have an api, you can just create a slice in Go that has all of the HTTPS clone URLs (Gitea can’t mirror SSH URLs), and feed them into that last loop.

You can see the full Gist of code here: https://gist.github.com/techknowlogick/c2367e03baff9f16b3c5cc9d9a5d13ca

Note: I have also used the above to migrate all my repositories on GitLab to Gitea, and also from GitHub to GitLab. The pseudo-code is pretty much the same for both.


Back to posts