Week 2. Project management

  • By Joey van der Bie
  • week

2. Project management

Assingment

Build a personal site describing you and your final project.
Upload it to the class archive.
Work through a git tutorial.
Further details on:http://fabacademy.org/2019/docs/FabAcademy-Assessment/project_management.html

Files

The Git repository

Software

Git
Gitlab
SSH
Jekyll
brew
ruby

Git and Gitlab

I have been using versioncontrol software since 2005 and Git since (about) 2010. So I have basic experience with creating, pulling, pushing, commits, and most important braking and fixing repository trees. I recently created a Git quickstart guide with a student of my for our Gitlab environment at the Amsterdam University of Applied Sciences.

Setup repository

This quickstart guide can also be used for the Gitlab environment of the fabacademy. The difference is changing the gitlab.fdmci.hva.nl names and repository url to the name and url of the fabcloud gitlab.

I noticed Henk used a slightly different approach for the SSH key.
However this approach did not fully work on OSX. We encountered some problems, the main problem was the config of the default server location in the ssh config.
When we tested the config provided by henk with example command ssh -T git@gitlab.fabcloud.com , it did not work.
But using the alias defined in config ssh -T git@git did work. git is the alias defined in the config, so it is good that it works,
but it should also work with the url.
I have no idea why the url did not work and did not found out why it did not work unfortunately.
I helped the team with getting everything setup on the OSX computers.
I modified Henk is tutorial to now also work with OSX as can be read in the section “Steps wwith SSH for OSX”

Steps with SSH for OSX

Install git

You do not have to install Git on OSX, this is allready on your computer.

Generate ssh-key

CLI way to generate a key for gitlab.
It is reacomment to use a separate keys for each host/server.
It makes it easier to delete a key after a project is finished, without breaking all your SSH connections.
In this example a “fabacademy-gitlab” key pair is created:

Go to your .ssh directory, located in your HOME directory:

cd ~/.ssh

Create a file for you ssh key to be stored in.

touch fabacademy-gitlab

Now change the accessright so that you only are allowed to see and edit the file:

sudo chmod 700 fabacademy-gitlab

Generate a new SSH key

ssh-keygen -t rsa -b 4096 -C "your@mailaddr.ess"

Runnig the command generates output like this:

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/*your user name*/.ssh/id_rsa): /Users/*your user name*/.ssh/fabacademy-gitlab
/Users/*your user name*/.ssh/fabacademy-gitlab already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /Users/*your user name*/.ssh/fabacademy-gitlab.
Your public key has been saved in /Users/*your user name*/.ssh/fabacademy-gitlab.pub.
The key fingerprint is:
SHA256:x+Psdc+tCgggggggUzi40penIuytdrDtxTbLeZQ "your@mailaddr.ess"
The key's randomart image is:
+---[RSA 4096]----+
|                 |
|                 |
|               ..|
|         .    . *|
|        S + .  B+|
|       +++*=*+*X |
|         o+B+O++*|
|         .=o+.o==|
+----[SHA256]-----+

Add your SSH key to the fabacademy git.

Copy your ssh public key using the command:

pbcopy < ~/.ssh/fabacademy-gitlab.pub

Go to your gitlab user ssh configuration: https://gitlab.fabcloud.org/profile/keys
Paste your public key in the “key” field
Provide a name for the key like: “your@mailaddr.ess fabacademy public key”
And select “Add key”

ssh-config

Edit you ssh config file with your favorite text editor.
It’s located in ~/.ssh/config

At the end of the file insert:

#Fabacademy git account
Host gitlab.fabcloud.org
HostName gitlab.fabcloud.org
User git
Identityfile ~/.ssh/fabacademy-gitlab
IdentitiesOnly yes

Git global setup

 git config --global user.name "your name"
 git config --global user.email "your@mailaddr.ess"

Create a new repository

git clone git@gitlab.fabcloud.org:academany/fabacademy/2019/labs/*yourlab*/students/*your-name*.git
cd your-name
touch README.md
git add README.md
git commit -m "add README"
git push -u origin master

Using an existing repository

cd existing_folder
git init
git remote add origin git@gitlab.fabcloud.org:academany/fabacademy/2019/labs/*yourlab*/students/*your-name*.git
git pull origin master
touch README.md
git add README.md
git commit -m "add README"
git push -u origin master

Adding and commiting changes to gitlab

When you have setup your git repository you can start submitting your new files and changes. From the root of your git repository you can use the following commands:

git pull (pulls the lastest changes from the gitlab repository)

git add . (Always run this command when you want to commit new files)

git commit . (Run commit to commit your changes, and provide a short but clear description)

git push ( this pushes the changes or additions to the gitlab repository)

Git submodules

As stated I know my basic Git for years, still there is always something left to learn, and this time it is to integrate to completely different git repositories into one while retaining the history. First before discussing how to do this, you should ask IF you NEED to do this. In most situations separate repositories represent separate filosofies and separate licencies. Merging this in one repoistory can cause licencing issues, and will probably cause update issues in the future (due to the difference in the trees). If you can at all avoid merging two separate repositories, please do. For example, one repository can use the other repository as a library. In this case however I was not allowed to have multiple Git repositories on the Gitlab fabcloud server, so I had to merge the two repositories into one. There are many ways to merge two repositories, each with their own advantages and thisadvantages. I searched the git documentation and stackoverflow discussions untill I found an approach I found the most elegant for my situation. I had the following requirements:

  1. preserve both history
  2. being able to retain different licencies of the two modules
  3. being able to merge changes from the original repository
  4. use one existing repository as a base

This sounds all very complex, but fortunately Git supports this out of the box via submodules. Basically a submodule is a separate Git repository in another Git repository. More details are explained on the official Git site. A submodule can be added to the originial repository via the command

git submodule [--quiet] add [<options>] [--] <repository> [<path>]

In my case it looked like this

 git submodule add https://github.com/lucasmbraz/lucasmbraz.github.io

Jekyll

I haven’t worked with Jekyll before, so my assingment for this week is to learn (by using) Jekyll. Jekyll is a framework on top of Ruby that allows you to create a website(with HTML/CSS and JavaScript) from Markdown and some config files. It has a vast library of themes, with very diverse functions: e.g. website, presentation, ect. My approach for learning Jekyll was:

  1. Install Jekyll
  2. Install a theme I like
  3. Modify theme and generate personal website

Install Jekyll

The Jekyll website has a tutorial for installing Jekyll How to install Jekyll and the prerequires for OSX https://jekyllrb.com/docs/installation/macos/ For the installation you need the Xcode command-line tools to be installed. I allready had them on my computer, so I could continue to the next step. As explained on the official site, Jekyll runs on Ruby, it is allready on my OSX version. Jekyll recommends to have at least Ruby v 2.2.5 I checked my version with command:

ruby -v

My version is: ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin16] To old, so time to update. The tutorial explains that with rbenv you can manage multiple versions of Ruby. I am currently not using Ruby, so I chose not to use rbenv, but just the default lastest install served from Homebrew.

brew install ruby

Ofcourse Homebrew did not updated my Ruby, but installed a new version next to the default OSX version. The lasy fix is to change the default path of Ruby to the new location. To not break things I changed the config path to new Ruby location

export PATH=/usr/local/opt/ruby/bin:$PATH

Checked correct config and versions with

gem env
ruby -v

Using all the correct paths (from user directory) and new Ruby version is: ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-darwin16]

Having Ruby setup, its time to install Jekyll using the page How to install Jekyll

gem install bundler jekyll

To test if Jekyll worked as expected I created a temporary folder and in that folder created a new Jekyll site.

jekyll new myblog

Navigated into the new blog, and try to build the site with command:

bundle exec jekyll serve

Which gave the error

Traceback (most recent call last):
	2: from /usr/local/opt/ruby/bin/bundle:23:in `<main>'
	1: from /usr/local/Cellar/ruby/2.6.0_1/lib/ruby/2.6.0/rubygems.rb:302:in `activate_bin_path'
/usr/local/Cellar/ruby/2.6.0_1/lib/ruby/2.6.0/rubygems.rb:283:in `find_spec_for_exe': Could not find 'bundler' (1.17.2) required by your /Users/biejh/Documents/HvA/Waag Cursus 2019/week2/myblog/Gemfile.lock. (Gem::GemNotFoundException)
To update to the latest version installed on your system, run `bundle update --bundler`.
To install the missing version, run `gem install bundler:1.17.2`

I am confused by the error, since gem installed the latest version of bundler (bundler-2.0.1.gem) and jekyll. The Jekyll install did not specify a specific bundle version. I checked with Micky and she also installed the bunlder 2.0 version but did not got the error. Only difference I can think about is the difference in OSX version. I am running Sierra, Micky High Sierra. Fortunately my problem was easily fixed by installing the older bundler version.

After succesfully installing Jekyll I hereby present my first Jekyll test page. Found on http://127.0.0.1:4000/ My first Jekyll test page

Install a Theme I like

Next, select a nice theme. After reading the url https://jekyllrb.com/docs/themes/ I still did not understand how to install and use a theme I found in the theme repository. I understand themes are gems, or could be gems, but how to install and use that gem is still unclear. Heidi and Micky told me to use the following steps:

  1. Download a theme
  2. Go into the folder
  3. Use jekyll build && jekyll serve
  4. It works! Magic!

While these steps work indeed for the first theme I downloaded. I still am not fully understanding Jekyll. After discussing with Henk, Micky and Heidi, I still don’t understand the structure of Jekyll.

Reading in the theme folder found that there is a structure most themes use, with folders as _post ect.
Also everything of the theme is defined in the _config.yml.
Information for the config can be found on this page https://jekyllrb.com/docs/configuration/default/

Yes! I finally start to understand.

Tried out theme Hyde.
Build failed because Jekyll was missing gems as paginate.
And I could not install them using the default “gem install …”
When looking in the gem folder paginate was present.
After some thinking found that my terminal was reverted to the old version of Rubys 2.0
Exported the path again to the new version and the missing gems was solved (almost).
Installed the missing gems and tried to build.
Turned out the theme was outdated and there wasn’t a quick fix since the used gems do not all work on my version of Jekyll 2.6.
It is unfortunate Jekyll does not support settings for themes to define for witch version of jekyll and gems it is designed.
Onwarts to find a new theme for my version of Jekyll…
Looked through the issues of the Jekyll Git and found many people having the same issues.
Many issues have to do with dependencies and deprications. Fortunately a jekyll expert forked the theme and made a gem version of the Hide theme.
This means all the dependencies are included and using bundle I can easily install the theme.
I downloaded the fork from: https://github.com/yyyc514/hyde/
And used the following command to build the theme:

bundle exec jekyll serve

Having succefully tried out a theme, it is now time to chose the final theme.
I found a nice fork of Hyde called Mickey.
Unfortantely Mickey also is not available as a gem, and after trieng uses outdated libraries, making it impossible to use with Jekyll 3 and up.
Fortunately it was forked by git user lucasmbraz and made compatible with Jekyll 3

It still wasn’t a gem,
so I had to build it using the following commands

  
  jekyll build
  jekyll server

Fortunately through all my trying with other themes I installed many different Jekyll modules resulting in a succefull build out of the box.
And voila a working version of theMickey theme for Jekyll

Important pitfalls:
Jekyll does not have a variable that theme builders can use to indicate for which version of Jekyll and modules it is made. This makes it very difficult for (new) users to select a team that is compatible with their build, or to create a build that is compatible with the theme.
This problem apperently does not occur when a theme is presented as a gem, but unfortunately the majority of themes presented on the Jekyll page appear to not be gems.
=

Modify theme and generate personal website for my actual FabAcademy repository

After having selected a working theme I had to integrate the theme in my allready existing FabAcademy(fabcloud) repository.
I started by inporting the Git repository of the them in my fabcloud repository. How I did this is explained in the section of git submodules.
Next I modified the theme to use my original markdown files and docs folder
Followed by customisiation of the layout and colors.
And last I modified the Gitlab config to serve my new website.
I explain the details in the following sections.

Using the original markdown files and docs folder

To keep on working in the docs folder from the template, I created symbolic links from the folders to the Jekyll theme.


  ln -s ../docs/assignments/ _drafts 
  ln -s ../docs/about/ about
  ln -s ../docs/images/ assets/images/posts

Unfortunately Jekyll does not allow symoblic links in the assets folder.

jekyll 3.8.5 | Error:  No such file or directory @ rb_sysopen - .../lucasmbraz.github.io/assets/images/posts

I tried to fix this by building Jekyll in production mode. According to the website this should work.
Unfortuntately this also did not work, Jekyll gave the same error.
As I was running out of time, I had to create a quick fix.
For now I will be copying the images by hand using the command:

cp -Rp ../docs/images/ assets/images/posts

To prevent the Git from duplicating the images, I put a Git ignore on the posts folder.

Collections

I wanted to use markdown files without dates that are needed for the posts markdown files.
The Jekyll documentation recommends to use pages. It was unclear to me how to implement, so I searched for help online, but found that collections or drafts are more common sollutions.

Collections are what is says, a collection of pages within a certain topic.
Again I read the collections doc page, but could not understand how to use this.
I found this help online and followed its advice.
Copied contents of doc file to the theme folder in the folder _pages.
Modified my config file with the following code:

  
  safe:					false
  include:				["_pages"]
  collections:
    pages:	
      output:				true
      permalink:			/:collection/:path/

Unfortunately that did not work. Nothing was visible.

Drafts

So I went to my last resort: drafts.
Copied everything in the _drafts folder and build jekyll with

  
  jekyll serve --drafts

And it worked! My markdown files in the template

Making it pretty

To make the theme more me, I changed the name, images and colors to something that I normally also used.
And added my avatar.
Most changes could be made in the _config.yml and the main.css in the assets folder.

Also added a hidden feature, to only show posts that are finished.
In the top config of a markdown I can now write


  hidden: true

And with the code


        \{\% if post.hidden != true \%\}
        ....
        \{\% endif \%\}

I filter out the pages on the main index.html and in the navigation menu html page.

Gitlab config for Jekyll

Last I had to tell Gitlab to use the newly generated theme in the CI module.
As explained by Henk the CI module is configures using the yml config file in the root of your repository .gitlab-ci.yml
I replaced the file contents with the following code:

  
  # This file is a template, and might need editing before it works on your project.
  # Full project: https://gitlab.com/pages/plain-html
  pages:
    stage: deploy
    script:
    - mkdir .public
    - cp -r * .public
    - mv .public public
    artifacts:
      paths:
      - public
    only:
    - master

The structure of the config file is explained at the gitlab pages site. and how to use it here. It basically copies the site folder “public” to the correct location.
I had to make a few modifications to make everything work.
First of, I was working in a different branch than the ‘master’ branch, so I changed the ‘only’ parameter to my current branch ‘jekylltheme’.
Also it specified in the Config->CI/CD->General Pipeline Settings to use the ‘jekylltheme’ branch instead of the ‘master’ branch.
The last change I made was tot he Jekyll theme config. I specified the output to the folder ‘public’ in the root of my repository with the setting:

  
  # Server
  destination:      	 ../public

After committing the code, the pages job got activated and my page was visible on:
http://fab.academany.org/2019/labs/waag/students/josephus-vanderbie/
Unfortunately I could not get the permalinks to work correctly, so verything needs to be ‘hardcoded’ to the correct URL.
It still needed some tweeks to the config for specified sitepath:

 
  site:                'http://fab.academany.org/2019/labs/waag/students/josephus-vanderbie/'
  baseurl:             'http://fab.academany.org/2019/labs/waag/students/josephus-vanderbie/'
  permalink:           /:title.html

And in all the pages that reference the main page, change the references to the main index.html hardcoded to

http://fab.academany.org/2019/labs/waag/students/josephus-vanderbie//index.html

And then everthing seems to be working!

My new pretty site for FabAcademy

Proper Jekyll config

I found the config that works with gitlab and configured it for my repo. Unfortunately this still does not work because of the submodule structure I use.

image: ruby:2.6

pages:
  script:
  - gem install jekyll
  - jekyll build -s custommickeytheme -d public/
  artifacts:
    paths:
    - public
  only:
  - jekylltheme

Markdown

While writing this page I again got familier with Markdown. Markdown is a simple notation langauge to quickly write readable documents.
The great thing is that these notations are both readable for humans as computers, and easy to learn.
I have been using Markdown for a while now, only I am not a power user. I mostly use the following notations: headings, code, URLs and images.

Headings can be created using the #. The more ### the further down your heading is. So one # is heading level 1, while 2 ## is heading level 2, ect.

Code can be written using 3 ``` then some code and end again with 3 ```

URLs using [here goes your text](here goes your URL)

And images are almost identical to URLs, only with a !, ![alt text](URL to image)

A breakline you make by using two spaces “ “

Then last of my commonly used notations, I use the escape, which is a \. It allows you to escape the Markdown notations and use the characters as is.

Reflection

While I certainly learned alot about Jekyll, I would not recommend this tool to anyone.
You can only use Jekyll in exactly the form as it is designed, if you want to modify its structure, you are on your own.
While you can create nice themes, the switch to gems makes that there is a lot of outdated material around.
You can compare

Share this story