Skip to content

(2026.03 and earlier) Authenticated users can read arbitrary files on the cluster‐manager host via symlink in Auth.GetUserPrivateKey

Julian Allende Russek edited this page Jul 9, 2026 · 16 revisions

Bug description

The Auth.GetUserPrivateKey API reads SSH private key files without symlink protection. An authenticated user can replace their ~/.ssh/id_rsa file with a symbolic link pointing to any file on the cluster-manager host (e.g., /etc/shadow, other users' SSH keys), and the API will return the target file's contents. This allows arbitrary file read on the cluster-manager instance by any authenticated RES user.

Affected versions

2026.03 and earlier

Mitigation

For 2025.06, 2025.06.01 and 2025.09:

  1. Download patch.py and symlink-file-read-fix.patch for version 2025.06, 2025.06.01, or 2025.09 by replacing with the directory path to download both the patching script and patch file. with the name of your RES environment, and with the RES version you have deployed in the command below:
    1. The patch applies to RES 2025.06, 2025.06.01, or 2025.09
    2. The patch script requires
      1. AWS CLI v2 — verify: aws --version
      2. Python ≥ 3.9.16 — verify: python3 --version
      3. Boto3 — install: pip3 install boto3 verify: pip3 show boto3
      4. setuptools < 70 — install: pip3 install verify: pip3 show setuptools
    3. Configure the AWS CLI for the account / region where RES is deployed, and make sure that you have S3 permissions to write to the bucket created by RES.
  OUTPUT_DIRECTORY=<output-directory>
  ENVIRONMENT_NAME=<environment-name>
  RES_VERSION=<res-version> # e.g. 2025.06, 2025.06.01, or 2025.09

  mkdir -p ${OUTPUT_DIRECTORY}
  curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch.py --output ${OUTPUT_DIRECTORY}/patch.py
  curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/symlink-file-read-fix.patch --output ${OUTPUT_DIRECTORY}/symlink-file-read-fix.patch
  1. Navigate to the directory where the patch script and patch file are downloaded. Run the following patch command:
python3 ${OUTPUT_DIRECTORY}/patch.py --environment-name ${ENVIRONMENT_NAME} --res-version ${RES_VERSION} --module cluster-manager --patch ${OUTPUT_DIRECTORY}/symlink-file-read-fix.patch
  1. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.
INSTANCE_ID=$(aws ec2 describe-instances \
                  --filters \
                  Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
                  Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
                  --query "Reservations[0].Instances[0].InstanceId" \
                  --output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}
  1. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name -cluster-manager-asg. Wait until the new instance is launched successfully.

For 2025.12, 2025.12.01 and 2026.03:

  1. Create an S3 bucket with ACLs disabled. If you already have an S3 bucket with ACLs disabled, you can skip this step. Otherwise, create one using either the console or the CLI:
    1. Option A (AWS Console)
      1. Follow the Creating a bucket (https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html) guide in the Amazon S3 User Guide. When configuring the bucket:
      2. Enter a bucket name and select the region where RES is deployed.
      3. Under Object Ownership, keep the default setting ACLs disabled. (Bucket owner enforced)
      4. Configure the remaining settings as needed (block public access, versioning, etc.)
      5. Click Create bucket.
    2. Option B (CLI)
      1. Replace <BUCKET-NAME> and <res-environment-aws-region> with your values and run the command:
aws s3api create-bucket \
    --bucket <BUCKET-NAME> \
    --region <res-environment-aws-region> \
    --create-bucket-configuration LocationConstraint=<res-environment-aws-region> \
    --object-ownership BucketOwnerEnforced

Note: If your region is us-east-1, omit the —create-bucket-configuration parameter.

  1. Download patch_host.py and idea-cluster-manager--.tar.gz by replacing with the directory path to download the patch script, with the name of your RES environment, with the name of a ACLs disabled S3 bucket under the account/region where RES is deployed, RES_VERSION with 2025.12, 2025.12.01, or 2026.03 in the command below, and , with Classic or GovCloud:
    1. The patch applies to 2025.12, 2025.12.01, 2026.03
    2. Configure the AWS CLI for the account/region where RES is deployed with S3 write permissions to the RES bucket.
      1. The patch script requires AWS CLI v2 and Python 3.9.16 or above, and boto3.
        1. AWS CLI v2 — verify: aws --version
        2. Python ≥ 3.9.16 — verify: python3 --version
        3. Boto3 — install: pip3 install boto3 verify: pip3 show boto3
    3. Set , , <RES_VERSION> , , according to your RES deployment and run the following commands to download the correct patch files into your local environment:
OUTPUT_DIRECTORY=<output-directory>
ENVIRONMENT_NAME=<environment-name>
RES_VERSION=<RES_VERSION>
BUCKET_NAME=<bucket-name>
PARTITION=<partition>

mkdir -p ${OUTPUT_DIRECTORY}
HASH=$(case $RES_VERSION in "2026.03") echo 19d5d5da;; "2025.12.01") echo 36092408;; "2025.12") echo 05a5f4b5;; esac)
curl https://research-engineering-studio-us-east-1.s3.us-east-1.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch_host.py --output ${OUTPUT_DIRECTORY}/patch_host.py
curl https://research-engineering-studio-us-east-1.s3.us-east-1.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz --output ${OUTPUT_DIRECTORY}/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz
  1. Before running the patch, note down the current package URI for rollback purposes. Save this value you will need it if you want to rollback later.
aws dynamodb get-item \
      --table-name ${ENVIRONMENT_NAME}.cluster-settings \
      --key '{"key": {"S": "cluster-manager.app_package_uri"}}' \
      --query 'Item.value.S' --output text
  1. Run the following patch command:
python3 ${OUTPUT_DIRECTORY}/patch_host.py --environment-name ${ENVIRONMENT_NAME} --module cluster-manager --zip-file ${OUTPUT_DIRECTORY}/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz --s3-bucket ${BUCKET_NAME} --partition ${PARTITION}
  1. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.
INSTANCE_ID=$(aws ec2 describe-instances \
    --filters \
    Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
    Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
    --query "Reservations[0].Instances[0].InstanceId" \
    --output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}   
  1. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name <res-environment>-cluster-manager-asg . Wait until the new instance is launched successfully.

Rollback

For 2025.06, 2025.06.01 and 2025.09: The patch script works as a toggle. Running the same command again will revert the patch. To rollback, re-run the exact command from the patching step:

If you're running this in a new terminal session, make sure to set the variables again:

OUTPUT_DIRECTORY=<output-directory>
ENVIRONMENT_NAME=<environment-name>
RES_VERSION=<res-version>

python3 ${OUTPUT_DIRECTORY}/patch.py --environment-name ${ENVIRONMENT_NAME} --res-version ${RES_VERSION} --module cluster-manager --patch ${OUTPUT_DIRECTORY}/symlink-file-read-fix.patch

For 2025.12 and later: The patch modified the DynamoDB cluster-manager.app_package_uri record to point to the patched tar.gz file in your S3 bucket. To rollback:

  1. Retrieve the original package URI that you saved in Mitigation Step 3. If you didn't save it, check CloudTrail or DDB item history for the previous value of the cluster-manager.app_package_uri key in the {environment-name}.cluster-settings table.
  2. Restore the original value:
aws dynamodb update-item \
     --table-name ${ENVIRONMENT_NAME}.cluster-settings \
     --key '{"key": {"S": "cluster-manager.app_package_uri"}}' \
     --update-expression "SET #val = :val" \
     --expression-attribute-names '{"#val": "value"}' \
     --expression-attribute-values '{":val": {"S": "<original-s3-uri>"}}' 
  1. Terminate the cluster-manager instance so the ASG launches a new one that picks up the original package.

Clone this wiki locally

⌂ 换网址