Git Repositories#

ICE ClusterWare™ head nodes can host Git repositories containing code or configuration files needed by compute nodes. These Git repositories can be used to enable compute nodes to run Ansible during boot up or to pull down version-controlled config files. The ClusterWare platform can directly host the Git repository on the head node, storing the files locally and allowing for modifications by local developers. Alternatively, the ClusterWare head nodes can be leveraged as a synchronization point for tracking upstream repositories. In the latter case, development work is primarily done on the upstream Git repository and ClusterWare tools can be used to sync or pull any changes down to the local copy. In both cases, the Git repositories are exported into the cluster for use with Ansible, Puppet, or as repositories of configuration files for initialization and boot-up.

Initial Preparation#

To facilitate interaction with the ClusterWare Git system, first add your personal public key to your ClusterWare admin account. This key is populated into the root user's (or _remote_user's) authorized_keys file for a newly booted compute node. The key is then used to provide SSH access to the Git repository.

To add the key to the admin record, run the following command:
```
scyld-adminctl up keys=@/full/path/.ssh/id_rsa.pub
```
Add the localhost's host keys to a personal known_hosts file with the following command:

Note

Adding the localhost’s host key is not strictly necessary, but adding it avoids an SSH warning that can interrupt scripting.
```
ssh-keyscan localhost >> ~/.ssh/known_hosts
```

Locally Hosted Repositories#

The scyld-clusterctl tool is used to create, update, and delete gitrepos objects in the ClusterWare database.

To create a new gitrepos object:

scyld-clusterctl gitrepos create name=gtest

To view the details of the gitrepos object:

scyld-clusterctl gitrepos ls -l
Git Repos
 gtest
  archive
    content
      chksum: sha1:b0b06c1d5acbcfc1e5a38314f31af6f3d8b4b63b
      filename: 50a2cc42303f4bf98c71c09b9e0d2adf
      mtime: 2024-08-02 18:41:43 UTC (0:42:47 ago)
      size: 12.0 KiB (12288 bytes)
  last_modified: 2024-08-02 18:41:43 UTC (0:42:47 ago)
  name: gtest
  sshgit: cwgit@<HEAD>:gtest
  url: <BASE_URL>/git-http/gtest

The main field of interest is sshgit, which can be used in Git operations. For example, to clone a copy of the gitrepos object:

git clone cwgit@192.168.122.88:gtest
Cloning into 'gtest'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (3/3), done.

By default, a single file is added in the otherwise empty repository: CwReadMe.txt. At this point, the repository on the local disk can be used like any other Git repository. For example, to create a new file in the directory:

cd gtest/
ls
CwReadMe.txt
echo "new file" > NewFile.txt
git add NewFile.txt
git commit -m "add new file"
[main 89d5f44] add new file
1 file changed, 1 insertion(+)
create mode 100644 NewFile.txt

Since authentication occurs via SSH keys, you can push the changes back to the ClusterWare-hosted Git repos:

git push
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 286 bytes | 286.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To 192.168.122.88:gtest
 d5a1a0b..89d5f44  main -> main

The “url” includes an HTTP-based URL that can also be used for Git download operations, but does NOT allow new data to be pushed up to the ClusterWare head node.

Mirroring Upstream Resources#

In many cases, ClusterWare Git repositories are used to mirror upstream repos. Developers can work with standard tools and interact with an enterprise Git service, such as GitLab or GitHub, without needing direct contact to the cluster. The ClusterWare platform can synchronize the locally-hosted copy with the upstream repository.

When creating the gitrepos object, specify a name, an upstream source, and a mapping for local-to-upstream branches:

scyld-clusterctl gitrepos create name=gtest2 \
        upstream=http://192.168.122.88/api/v1/git-http/gtest \
        branch_map=main:main

In this example, only one branch is defined and it shares the same name (main) in the local and upstream repos. For more complex configurations, a comma-separated list of local-to-upstream mappings can be provided, and the local/upstream names do not need to be the same. For example: “main:main,cw_dev:up_dev” defines two mappings. The first maps the local “main” to upstream “main” and the second maps local “cw_dev” to the upstream “up_dev”.

Since the ClusterWare platform creates a minimal Git repository by default (with a CwReadMe.txt file), it is often necessary to “sync” and “sync reset” so that the local repos downloads the relevant metadata from the upstream repos, and then pulls down the current upstream contents. For example:

scyld-clusterctl gitrepos -igtest2 sync
Git Repos
 gtest2
 Branch 'main' is 1 commit ahead of and 3 commits behind branch 'upstream/main'

scyld-clusterctl gitrepos -igtest2 sync reset
Git Repos
 gtest2
 Branch 'main' reset to branch 'upstream/main'

Note

This extra “reset” is caused by the placeholder content that ClusterWare v.12.4.0 creates. This may change in future releases.

When new upstream changes are made, pull the changes to the local copy using the following command:

scyld-clusterctl gitrepos -igtest2 sync pull
Git Repos
 gtest2
 Branch 'main' reset to branch 'upstream/main'

Next, re-sync any clients that might need that new data (for example, all of the compute nodes):

git pull

Public Access#

For development and testing of the config files and code in a Git repository, authenticated access is needed so that the git push commands work. The ClusterWare platform can also provide public access to any gitrepos hosted on the head nodes.

The base URL for public access is found in the scyld-clusterctl output:

scyld-clusterctl gitrepos ls --fields url -l
Git Repos
 gtest
  url: <BASE_URL>/git-http/gtest

For the above example, public access can be found at:

git clone http://192.168.122.88/api/v1/git-http/gtest
Cloning into 'gtest'...
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 6 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (6/6), done.

In this example, if another file is added and pushed back to the ClusterWare gitrepos object, the push will fail. Even if SSH keys have been loaded, this is using a public URL, which disallows upload of new content:

echo "another file" > AnotherFile.txt
git add AnotherFile.txt
git commit -m "adding another file"
[main b900bf1] adding another file
 1 file changed, 1 insertion(+)
 create mode 100644 AnotherFile.txt
$ git push
fatal: unable to access 'http://192.168.122.88/api/v1/git-http/gtest/': The requested URL returned error: 403

If public access to a gitrepos needs to be disabled for some reason, setting the “public” field to False blocks access:

scyld-clusterctl gitrepos -igtest update public=False