PR version of envVarGroup is missing keys that have sync: false

Hi all. I have an env group defined in my render.yaml. Something like this:

previewsEnabled: true

envVarGroups:
  - name: staging
    envVars:
      - key: ELIXIR_VERSION
        value: 1.11.4
      - key: MAILGUN_API_KEY
        sync: false

The group is created, I set the MAILGUN_API_KEY value manually in the web UI. Everything was working OK. For preview environments, the group was copied correctly and MAILGUN_API_KEY was included with the value I set manually. This is the desired behavior.

Then I created another PR and changed the value of the ELIXIR_VERSION env var there and the env var group copied for that PR doesn’t include the MAILGUN_API_KEY env var (or none with the sync: false set). This is not desired and I am wondering if it’s some bug or is it the intended behavior that I have just discovered?

Thanks a lot for your help.

Hi Stefan,

When you view this PR in the Github UI, and click on the “files changed” tab, do you see the expected render.yaml content there?

Yes, it’s there.

Does your render.yaml include a reference to the env group?

services:
- name: my-service
  envVars:
  - fromGroup: staging

If you already have that “fromGroup” parameter set, then you might need to try defining the env group in the UI, but still referencing it in the render.yaml file. Does that work for you?

I’ll try to create simplest possible repo to replicate this bug and post a link here.

OK, I got it.

render.yaml
https://github.com/smarkup/render_plug/blob/main/render.yaml

Service ID
srv-c2jc9bu31hs87nrl3d9g

Steps I did:

  1. Render web UI → YAML → New From YAML → selected the repo mentioned above
  2. EXAMPLE_ENV_VAR_GROUP was created but without the NOT_SYNCED_ENV_VAR - that is what I expected
  3. added NOT_SYNCED_ENV_VAR with some arbitrary value into the EXAMPLE_ENV_VAR_GROUP
  4. created a PR on the render_plug repository
  5. preview environment was created along with EXAMPLE_ENV_VAR_GROUP PR #1, but that doesn’t contain the NOT_SYNCED_ENV_VAR

Expected result:
EXAMPLE_ENV_VAR_GROUP PR #1 to contain the NOT_SYNCED_ENV_VAR with the value I set manually in the Render web UI.

Thanks a lot for your help, @Aaron_Gruberg :slight_smile:

Can you try removing this section envVarGroups from your yaml, and setting this env group in the UI via the Env Groups page? This may be necessary for env groups that contain variables set to sync: false.

If I remove the envVarGroups section from my render.yaml file, there will be no sync: false, and then it will become completely different scenario.

I read the docs and the information there regarding sync: false seems to be contradicting your advice.

Environment groups

  • Environment groups can be used to propagate sync: false environment variables to preview environments.

Or maybe I don’t understand the sentence I am quoting above. Could you please elaborate a bit?

I don’t want to set my env groups through UI, because it won’t be an infrastructure as code anymore. I want to keep it in the render.yaml to track changes in GIT among other things.

Sorry for the confusion there, environment groups are actually used to replace sync:false variables. This is necessary because sync:false requires manual input, while the PR previews are automatic. The environment group can still be referenced in your yaml as you are already doing, it’s just not created in the yaml.

    envVars:
      - fromGroup: staging

I need to summon this thread. I am still fighting this problem and it’s extremely frustrating. It seems that the docs are updated (see picture below) but not correct or I am unable to understand it.

Steps I did

  1. Render UI → YAML → New from YAML, select my repo
  2. Fill the value of the ENV_VAR_SYNC_FALSE env var when asked
  3. Done

Result

  • :white_check_mark: Service was created.
  • :white_check_mark: Env var group was created with the ENV_VAR_SYNC_FALSE set to value I used in the step 2 above.

But then I created a PR.

Result

  • :x: ENV_VAR_SYNC_FALSE is not present in the linked PR version of the envVarGroup specified in my render.yaml file.

Main service: srv-c3s65qqp7h3vcb5ep76g
PR service: srv-c3s6g0o41tf4qrnqln2g

Screenshot from docs:

Please help. :pray:

Hi Steven, it looks like you used the Render UI > YAML option. The recommended step is to use the Render UI > Env Groups feature.

Once that part is done, the environment group can be referenced in your yaml as you are already doing, it’s just not created in the yaml.

    envVars:
      - fromGroup: staging

Thanks Arron, in that case, I have a few questions:

  1. shouldn’t this be mentioned in the docs?
  2. in my opinion it goes against the idea of infrastructure as code, doesn’t it?
  3. is there a plan to support creating the group through render.yaml?
  4. what happens if
    4.1 I create an env group with some env vars via the UI
    4.2 I add a new env var with sync: false via the render.yaml file to that group
    4.3 I fill it out in the UI
    4.4 I create a PR, will that new env var be copied to the PR version of that env group?

Thanks a lot.

Good point, our section on environment groups is pretty brief, but we do have one here. YAML Specification | Render · Cloud Hosting for Developers

Render environment groups are still referenced in your YAML file, which is still IaC. I didn’t see an open feature request for creating the group in YAML but you can create one here! Feature Requests | Render

The part you mention in 4.2 "add a new env var with sync: false via the render.yaml" wouldn’t really be recommended. Instead of creating a sync: false env var in the render.yaml, we suggest using the Env Group UI feature for sync: false vars. The environment groups section of our docs has a little more background on this.

Thank you for your explanation Aaron.

There’s no mention of the fact, that the env group in render.yaml file should be first created through the UI. That was the thing that I thought should be noted in the docs, because it doesn’t make sense to me.
Also the fact that I was able to create it with render.yaml didn’t suggest that I am doing something incorrectly.

To be honest, I still feel that I am missing something :sweat_smile:, because what would be the point of declaring an env var group in the render.yaml file if it has to be created in the UI first?

BTW: I know about the feedback.render.com site, I already created few feature requests there, some of them are already implemented or are in progress. :slight_smile:

1 Like

Hi Stefan, sorry for missing some of your earlier points. I was able to repro what you’re describing, here’s what I found.

Our PR Preview deploys currently copy everything contained in the YAML file, but do not copy anything that was entered manually, which is why the PR Preview was not copying the sync:false env variable whose value you entered manually in the UI.

I chatted with one of our engineers who agreed that most folks would expect PR Previews to copy both the YAML contents and any environment details that have been manually in the UI. We’ll share these findings with our product team to help them consider any changes that might be helpful in the future. Thanks for including all your repro steps and sharing so much detail about what you tried!

Thanks a lot Aaron.

Hi Aaron,

Do the .env secret files currently get copied over to preview envs? That might help as a workaround until the env vars are fixed.

EDIT: Looks like that does not work either.

We really don’t want to hardcode secrets like our Stripe API Keys in a render.yaml, but it seems that there’s no other way of using preview environments?

Is there an ETA for the fix on this?

Hi @rattrayalex,

One workaround that you could use would be to manually create the Env Group and enter the values through the Render Dashboard. Then from your YAML, you should be able to reference the group by name and have the values copy over for your preview environments as well. Unfortunately, I understand this is a bit unintuitive and defeats the purpose of IaC. We do hope to address this behavior in the future to make this process more streamlined!

Okay, thanks Jade! We’ll give that a try.

Can we simply remove the EV groups from the render.yaml for this to work? Or do we have to recreate them entirely?

Hi @rattrayalex, if you have already created the group manually via the UI, you can remove the env group spec from your yaml and just reference the group name for the services you are associating to them like so:
fromGroup: my-env-group