Skip to content

Conversation

@Paritosh-Anand
Copy link

@Paritosh-Anand Paritosh-Anand commented Dec 10, 2025

Purpose

Support of IRSA IAM roles for service accounts is an important feature which helps in configuring the necessary access for the tablet-server and coordinater-server to interact with S3 for remote storage.

I see that helm chart for version 0.9.0-incubating is still not released. Hence, not bumping the version with this change. Let me know if this is fine.

Update: In my testing I figured that coordinater-server also tried to connect with S3 and failed to start if the authentication was missing.

Also it is important to note that we need to set few extra configurationOverrides to make it work

configurationOverrides:
    # Configure S3 as remote storage backend for snapshots
    # Format: s3://bucket-name/path-prefix
    remote.data.dir: s3://bucket-name/remote-data
    # S3 region (required for S3 access)
    # Update this to your actual AWS region (e.g., eu-west-1, ap-southeast-1)
    s3.region: ""
    # Workaround: Set dummy non-empty access key to bypass Fluss's delegation token mechanism
    # This prevents S3FileSystemPlugin.setCredentialProvider() from calling
    # S3DelegationTokenReceiver.updateHadoopConfig() which requires delegation tokens.
    # The credential provider chain below will then use IRSA credentials instead.
    # These dummy values will be ignored by Hadoop's credential provider chain.
    s3.access-key: "dummy"
    s3.secret-key: "dummy"
    # Configure S3A credential provider chain for IRSA
    # Fluss uses AWS SDK 1.12.319 and Hadoop 3.4.0
    # The chain will try WebIdentityTokenCredentialsProvider first (for IRSA),
    # then fall back to other providers if needed
    # IRSA credentials are read from environment variables set by EKS:
    # - AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
    # - AWS_ROLE_ARN=arn:aws:iam::ACCOUNT:role/IAM-ROLE
    # Note: The dummy access keys above prevent Fluss from using delegation tokens.
    # Hadoop S3A will use this credential provider chain which supports IRSA.
    fs.s3a.aws.credentials.provider: com.amazonaws.auth.WebIdentityTokenCredentialsProvider,com.amazonaws.auth.EnvironmentVariableCredentialsProvider,com.amazonaws.auth.InstanceProfileCredentialsProvider

Linked issue: close #2141

Brief change log

ServiceAccount for TabletServer

Tests

API and Format

Documentation

Copy link
Contributor

@rionmonster rionmonster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, approving! 👍

@Paritosh-Anand Paritosh-Anand changed the title [Helm] ServiceAccount for TabletServer [Helm] ServiceAccount for TabletServer, CoordinatorServer Dec 11, 2025
Copy link
Contributor

@dreger1997 dreger1997 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Helm standard for ServiceAccounts is using convention serviceAccount.create so I would recommend using

serviceAccount:
  create: false

also as stated in the comment where you say if create is true.

Beyond that I would make comments generic (not AWS specific).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Helm] ServiceAccount for TabletServer, CoordinatorServer

3 participants