SSH Keys in GitLab CI/CD: Why Double Quotes Matter & How to Fix Common Errors

Working with SSH keys inside GitLab CI/CD pipelines can be tricky, especially for newcomers to DevSecOps. Small syntax issues—like missing double quotes—or malformed key data often lead to confusing error messages such as “Invalid format” or “Error loading key … error in libcrypto.” This article explains the role of double quotes when handling SSH‑key variables, walks you through the most frequent GitLab CI errors, and provides step‑by‑step solutions so your pipelines run smoothly.

Table of Contents

1. Why Double Quotes Are Required for SSH Key Variables
2. Typical GitLab CI Errors Involving SSH Keys
   • 2.1 Invalid Format Errors
   • 2.2 Libcrypto Loading Errors
3. Step‑by‑Step Fixes
   • 3.1 Correctly Storing and Referencing Keys
   • 3.2 Testing the Key Inside a Job
4. Practical Example: Deploying with a Private Key
5. Tips & Best Practices
6. Common Questions

Why Double Quotes Are Required

Variable Expansion vs. Word Splitting

• With double quotes ("$SSH_KEY"):
  The shell expands the variable once and treats the entire content as a single argument. This preserves line breaks and spaces that are intrinsic to PEM‑formatted keys.

• Without double quotes ($SSH_KEY):
  The shell performs word splitting on whitespace. Each line of the key becomes a separate argument, breaking the PEM structure and causing the SSH client to reject the key.

What Happens Inside a GitLab CI Job?

script:
  - echo $SSH_KEY          # ❌ Splits on newlines → corrupted key
  - echo "$SSH_KEY" > /root/.ssh/id_rsa   # ✅ Correct, preserves format

Using double quotes ensures the private key is written exactly as it appears in the GitLab CI/CD variable, preventing “Invalid format” and “error in libcrypto” failures.

Typical GitLab CI Errors Involving SSH Keys

1. Invalid Format Error

Symptom:

ERROR: Invalid format

Cause:
The key stored in the CI/CD variable has been altered—usually by extra spaces, missing line breaks, or HTML‑entity conversion during copy‑paste.

Root Sources:

• Copy‑pasting from a web page that adds invisible characters.
• Saving the key in a text editor that strips trailing newlines.
• Not using the “Protected” flag for variables that need to be accessed by protected branches only.

2. Libcrypto Loading Error

Symptom:

Error loading key "/root/.ssh/id_rsa": error in libcrypto

Cause:
OpenSSL’s libcrypto cannot parse the PEM data because the file is malformed (often due to missing double quotes or truncated content).

Typical Trigger:
Running a script that references $SSH_KEY without quoting, leading to a broken file on the runner.

Step‑by‑Step Fixes

1. Store the Private Key Correctly

1. Open the key in a plain‑text editor (e.g., VS Code, Notepad++) and verify it starts with -----BEGIN RSA PRIVATE KEY----- and ends with -----END RSA PRIVATE KEY-----.
2. Copy the entire block, including line breaks.
3. In GitLab, go to Settings → CI/CD → Variables → Add Variable:
   • Key: SSH_KEY
   • Value: Paste the key exactly as copied.
   • Mask: ✅ (optional, hides the value in job logs)
   • Protected: ✅ (if only protected branches need it)
   • Environment scope: * (or limit as required)

2. Write the Key Inside the Job Using Double Quotes

deploy:
  stage: deploy
  image: alpine:latest
  script:
    # Create .ssh directory with proper permissions
    - mkdir -p /root/.ssh
    - chmod 700 /root/.ssh

    # Write the key – note the double quotes!
    - echo "$SSH_KEY" > /root/.ssh/id_rsa
    - chmod 600 /root/.ssh/id_rsa

    # Verify the key can be read
    - ssh-keygen -y -f /root/.ssh/id_rsa > /dev/null

If the ssh-keygen command succeeds, the key is correctly formatted.

Practical Example: Deploying a Service via SSH

stages:
  - build
  - deploy

build_job:
  stage: build
  script:
    - echo "Building Docker image..."
    - docker build -t myapp:${CI_COMMIT_SHA} .

deploy_job:
  stage: deploy
  image: ubuntu:20.04
  before_script:
    - apt-get update && apt-get install -y openssh-client
  script:
    - mkdir -p /root/.ssh && chmod 700 /root/.ssh
    - echo "$SSH_KEY" > /root/.ssh/id_rsa && chmod 600 /root/.ssh/id_rsa
    - ssh -o StrictHostKeyChecking=no user@my.server.com "docker pull myapp:${CI_COMMIT_SHA} && docker run -d myapp:${CI_COMMIT_SHA}"

Notice the double‑quoted $SSH_KEY and the explicit permission settings—both are essential to avoid format‑related failures.

Tips & Best Practices

• Always use double quotes when expanding multi‑line variables in Bash scripts.
• Validate the key locally with ssh-keygen -y -f <file> before committing it to GitLab.
• Enable “Mask” for secret variables to prevent accidental exposure in job logs.
• Keep the key file permissions tight (chmod 600) to satisfy SSH security checks.
• Test the pipeline in a protected branch first; this prevents accidental runs on the main branch with malformed keys.
• Document the exact copy‑paste steps (e.g., start at line 1, include the final newline) in your internal runbooks.

Common Questions

By understanding why double quotes are essential, correctly storing your SSH key in GitLab, and following the troubleshooting steps above, you’ll eliminate the most common SSH‑related CI/CD failures and keep your DevSecOps pipelines secure and reliable. Happy coding!