20200302-hugo_and_gitlab_ci_cd.draft (10514B)
1 --- 2 title: "hugo and gitlab ci cd" 3 date: 2020-03-02T18:50:01Z 4 summary: How I set up a Hugo website and deployed with Gitlab's CI/CD pipeline 5 categories: [workshop] 6 tags: [website, hugo, devops, gitlab, automation] 7 draft: false 8 --- 9 10 It is about time I get this weblog started again, it has been too long... 11 12 As I had decided to migrate my code repositories off Github it felt right I should move my weblog from being hosted on Github Pages. At first I was going to move to Gitlab Pages and keep everything the same, but **_where_** is the fun in that! 13 14 Instead of Jekyll, the static site generator [Hugo](https://gohugo.io) was recommended to me. I also wanted to play around with Gitlab's CI/CD pipeline tool, so thought the weblog migration would be a great project. I have migrated all my old posts to the new platform as well, so you won't miss out on anything! 15 16 ### new projects 17 18 Generate a new Hugo site, for this example I will be calling mine 'pyratelog' 19 ``` 20 hugo new site pyratelog 21 ``` 22 23 Navigate into the new directory and initialise it as a git repository 24 ``` 25 cd pyratelog 26 git init 27 ``` 28 29 Create a new project in Gitlab 30 31  32 33 Add your new Gitlab project as a remote repo to your Hugo site and make an initial commit if you want 34 ``` 35 git remote add origin git@gitlab.com:pyratebeard/pyratelog.git 36 git add . 37 git commit -m "initial commit" 38 git push -u origin master 39 ``` 40 41 Your Gitlab project should now be populated with a `config.toml` file and the 'archetypes' directory. 42 43  44 45 I won't keep mentioning when to commit changes to git as we all work differently. We will come to it a bit later when we configure our CI/CD pipeline. 46 47 ### configure hugo 48 49 Let us add a theme to our Hugo project, in this case I will use my own 'futuremyth' theme 50 ``` 51 git submodule add https://gitlab.com/pyratebeard/hugo-futuremyth.git themes/futuremyth 52 echo 'theme = "futuremyth"' >> config.toml 53 ``` 54 55 I have added in the 'paginate' variable to change the default of 10 items to 5, and also set a static directory for use with images in my log entries 56 ``` 57 cat >> config.toml << EOF 58 paginate = "5" 59 staticDir = ["static"] 60 EOF 61 ``` 62 63 I found it is a good idea to change some of the cache directories. There was an issue I had in my Gitlab CI/CD pipeline with root permissions being set on a directory, causing the pipeline to fail 64 ``` 65 cat >> config.toml << EOF 66 [caches.images] 67 dir = ":cacheDir/_gen" 68 [caches.assets] 69 dir = ":cacheDir/_gen" 70 EOF 71 ``` 72 73 You should also edit the 'baseURL' and 'title' variables in your `config.toml`. 74 75 You can start Hugo on your local machine in development mode using 76 ``` 77 hugo server -D 78 ``` 79 80 If you navigate to http://localhost:1313 you should see a fairly empty page. To add new content you run 81 ``` 82 hugo new posts/hello_world.md 83 ``` 84 You change the path to whatever you want, and it will be created under the 'content' directory. 85 86 If you left your deployment server running you should see that in your browser the site should automatically updates. You first entry should show the title of your post and the date. You can open the markdown file in your favourite editor and start writing below the second set of hyphens (`---`). Everything between the hyphens is metadata for the page. You can add more if you like, I add a 'summary', 'categories', and 'tags' in the following way 87 ``` 88 summary: How I set up a Hugo website and deployed with Gitlab's CI/CD pipeline 89 categories: [tech] 90 tags: [website, hugo, devops, gitlab, automation] 91 ``` 92 93 We can now build our site by running 94 ``` 95 hugo 96 ``` 97 98 This won't include our first post because we have left the `draft` variable as `true`. When you are ready to publish change it to `false` and build the site again. You can build with drafts included by running 99 ``` 100 hugo -D 101 ``` 102 103 ### autodevops 104 105 There a many ways you can host a website, and many ways you can use Gitlab's CI/CD pipeline to automate the process. The method I have opted for is to run my Hugo site in a docker container on a DigitalOcean droplet. I have chosen **_not_** to use `docker-compose` to include the Nginx reverse proxy as I host other things behind Nginx and don't want it to be restarted each time I post a log entry. 106 107 On a server with Docker already installed you can set up your Nginx reverse proxy with a Let's Encrypt companion to deal with SSL. 108 109 First, we need to create an new network 110 ``` 111 docker network create nginx-proxy 112 ``` 113 114 Then we can start the Nginx container 115 ``` 116 docker run -d --name nginx-proxy \ 117 -p 80:80 -p 443:443 \ 118 --net nginx-proxy \ 119 -v /etc/nginx/certs \ 120 -v /etc/nginx/vhost.d \ 121 -v /usr/share/nginx/html \ 122 -v /var/run/docker.sock:/tmp/docker.sock:ro \ 123 jwilder/nginx-proxy 124 ``` 125 126 Confirm the container is running by running `docker ps`, the output should look like this but with a different container id 127 ``` 128 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 129 ab7626dd1bec jwilder/nginx-proxy "/app/docker-entrypo…" 2 seconds ago Up 1 second 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp nginx-proxy 130 ``` 131 132 Next we can start the Let's Encrypt companion container (change the email address) 133 ``` 134 docker run -d --name nginx-proxy-letsencrypt \ 135 --volumes-from nginx-proxy \ 136 -v /var/run/docker.sock:/var/run/docker.sock:ro \ 137 -e "DEFAULT_EMAIL=youremail@yourdomain.tld" \ 138 jrcs/letsencrypt-nginx-proxy-companion 139 ``` 140 141 In the Hugo repository we need to create a `.gitlab-ci.yml` file so that we can harness the power of the CI/CD pipeline. 142 143 Enter the following in to the file 144 ``` 145 build: 146 stage: build 147 image: mapitman/docker-hugo:latest 148 services: 149 - docker:dind 150 before_script: 151 - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY 152 script: 153 - git submodule update --init --recursive 154 - docker build --pull -t $CI_REGISTRY_IMAGE:latest . 155 - docker push $CI_REGISTRY_IMAGE:latest 156 157 deploy: 158 stage: deploy 159 image: docker:latest 160 services: 161 - docker:dind 162 tags: 163 - deploy 164 before_script: 165 - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY 166 script: 167 - docker pull $CI_REGISTRY_IMAGE 168 - docker run -d --name "$CONTAINER_NAME" --expose 1313 --net nginx-proxy -e VIRTUAL_HOST=log.pyratebeard.net -e LETSENCRYPT_HOST=log.pyratebeard.net -v $(pwd):/src $CI_REGISTRY_IMAGE 169 ``` 170 171 You will notice some variables in the file beginning with `$CI_REGISTRY_*` and one called `$CONTAINER_NAME`. These are variables we declare in Gitlab. 172 173 If you use Multi-factor Authentication (MFA) on your Gitlab account you will need to generate a Personal Access Token to use in place of your password. To do this navigate to your account settings and under 'Access Tokens' fill in the Name and tick the 'api' scope. If you don't enter an expiry date the token will not expire. 174 175  176 177 Make sure you copy the access token, we will need it for the next step. 178 179 Navigate to your repository in Gitlab then to 'Settings', 'CI/CD', and expand the 'Variables' section. Enter the following Key/Value pairs 180 181 | Key | Value | 182 | --- | --- | 183 | CI_REGISTRY | registry.gitlab.com | 184 | CI_REGISTRY_IMAGE | registry.gitlab.com/_username_/_project_name_ | 185 | CI_REGISTRY_USER | _username_ | 186 | CI_REGISTRY_PASSWORD | _personal_access_token_ | 187 | CONTAINER_NAME | _anything_ | 188 189 Mark the `CI_REGISTRY_PASSWORD` variable as 'Protected' and make sure you click 'Save variables'. 190 191 Before we push our new Hugo project to Gitlab we need to configure a Runner. Gitlab Runners are used to execute the jobs in our pipeline. 192 193 At first I was trying to use a docker runner to build and deploy my project. Building a new docker image was easy, using kaniko, but I struggled to get the deploy section working. In the end I brought it right back to the Keep It Simple Stupid (KISS) philosophy. 194 195 On the server install a Runner following the instructions [here](https://docs.gitlab.com/runner/install/linux-repository.html). 196 197  198 199 Use the token that is shown in your repo CI/CD settings under 'Runners', add the tag 'deploy', and select the 'shell' executor. 200 201 Make sure you add the gitlab-runner user to the docker group 202 ``` 203 sudo usermod -aG docker gitlab-runner 204 ``` 205 206 Right, we are almost ready to go! The final file we need is a Dockerfile. This tells docker what we want our image to look like. Enter the following into your Dockerfile, changing the base URL as required 207 ``` 208 FROM jojomi/hugo 209 210 COPY . /src 211 WORKDIR /src 212 213 ENV HUGO_WATCH=true 214 ENV HUGO_THEME=futuremyth 215 ENV HUGO_BASEURL=https://log.pyratebeard.net 216 217 RUN hugo 218 ``` 219 220 Now publish your Hugo site by just running `hugo` again. Make sure all your changes are committed and push 221 ``` 222 git push 223 ``` 224 225 If you navigate to the CI/CD Pipelines page in your Gitlab project your should see the jobs being run. 226 227  228 229 Both jobs in the pipeline should complete successfully. Here is a breakdown of what the runner is doing: 230 - build phase 231 - building a new docker image containing our hugo project 232 - pushing the new image to our gitlab project's container registry 233 - deploy phase 234 - pulling our new docker image from the registry onto our server 235 - starting a new container using the image 236 237 There is one final thing we have to add to our `.gitlab-ci.yml` file to ensure the next time you push nothing breaks. In the deploy script, between the `docker pull` and `docker run` commands enter the following 238 ``` 239 docker stop $CONTAINER_NAME 240 docker rm $CONTAINER_NAME 241 docker rmi -f $(docker images --filter "dangling=true" -q) 242 ``` 243 244 These lines make sure to stop and remove the container with the name you have specified, docker doesn't like duplicates. The third line removes and old images to keep things tidy. 245 246 I hope this, fairly long, post helped you in someway. If you want to get any further information you can get in touch on [mastodon](https://mastodon.social/@pyratebeard), or any other way mentioned on my [home page](https://pyratebeard.net). 247