Create temporary resources on external services

Hello all,

What’s the best way to trigger external services setup / teardown with Render previews environment ?

I have multiple use cases :

  • Whitelisting the Render’s preview URL in Auth0 for auth
  • Create temporary SNS+SQS resources, or S3 buckets on AWS

Is it something that should be part of my build process using tools like CDK or Pulumi?
What about the teardown phase, how which event could I delete those resources when Render is deleting the temporary stack?

Great question!

For the startup side, you may be able to take advantage of the preview environment initialization feature. You can run a command from within the preview environment in a context where the URL is exposed as an environment variable. You can setup any external services required from there, before the service is marked as deployed.

I’m not sure we have a perfect solution for tearing down external resources. You could potentially use Environment Variable Overrides to use a single instance of external resources across all preview environments so that you aren’t creating a high number of services and running up costs, but that might not give you the environment isolation that you are looking for.

We are working on building out an API. With that, you would be able destroy external resources in response to service deletion events. That should be in early access soon.

If you have any thoughts on a different workflow that would work better for you, I’d love to hear it!

Hello jake,

The other solution is to use something like github actions which are capable of passing data between workflow steps (so I could get the current preview URL from render’s step and use it when the PR is closed to trigger my cleanup). It’s doable but not really an integrated solution. With the API it will be way easier I think.

That should be in early access soon as in Q1 2021? :smile:

I can’t make any promises on timelines, but we are targeting this quarter to roll out early access for some parts of the API.

I’m digging a little more into Render features. Right now the deploy url is great in term of usability and integration with Github actions, but I still need more. :grinning_face_with_smiling_eyes:

To be sure I understand the API capabilities for my use case, will I be able to deploy/suspend/resume/delete Pull Request Previews environments with it? With this 4 primitives I could support any workflow I want with Github Actions! :confetti_ball:

Hey @jgoux! We’re excited to see what you build by combining Github actions with the Render API.

The schema for the API isn’t finalized yet, but we’re planning to include endpoints that will let you deploy with a render.yaml file from any branch, and then suspend/resume/delete the resources defined in the file. You should also be able to suspend/resume/delete the resources created as part of a preview environment. Deploying preview environments on demand is not part of the current plan, since this behavior is controlled by the previewsEnabled field in the render.yaml file itself. Without too much work you should be able to reimplement similar functionality by turning off preview environments and using the API to deploy from the PR branch. It’ll be less convenient than our built-in preview environments, but with infinite customizability.

If you have specific flows in mind, I’d be happy to dive in deeper. Feel free to follow up here or in a DM.

Hello David,

If I understand correctly I think the workflow I have in mind would be possible with the API.

We’re using a continuous deployment git flow, so we only have one production branch (main) and then we open PR to implement features/bug fixes. As soon as a PR is merged into main, it’s deployed in production. As we have no layer between our PR and our production branch, it’s very important for us to be as qualitative as possible in each PR, which means having a strong test suite and preview environments mirroring the production environment to have as much confidence as we can in the code we produce.

We’re using a render.yaml to declare our whole infrastructure on Render, including the previewEnabled option.

What we miss today is a way to manually deploy the preview environments on demand, because there are a lot of situations where we don’t want it to be deployed as soon as the PR is opened :

  • Draft PR
  • PR that doesn’t need a preview environment (doc, tiny fixes…)

What we’d like is to control (with Github Actions) when the preview environment is deployed on each PR, and being able to suspend/resume/delete it too.
For example we could do it using comments on the PR :

In order to support this workflow, we would need these functionalities available when using the API :

  • Trigger a deploy/suspend/resume/delete preview environment based on the current PR
  • Query the current preview environment status
  • For the “resume” state, if we resume the preview environment and there were commits in between, I expect it to be up to date with the current PR code. And to be updated for each future push, exactly like how deploy works.

If I understand correctly, as you don’t plan to enable preview environment control with the API right now, we would have to declare 2 different render.yaml file? (one for the production and one for all the preview environments? Will the preview environments still be isolated from each other with this approach?)

Thanks for the extra detail @jgoux! This is very helpful in understanding your use case. As you and I discussed in More control over Preview Environment in Pull Request, this is a workflow that we plan to support natively, without the need for custom solutions based on the API. That said, the API may be available before we implement this finer-grained control over preview environments.

You asked some smart questions, and they made me realize that I need to amend what I said before about deploying with a render.yaml file. My earlier suggestion was to turn off preview environments and use the same render.yaml for both the production branch and PR branches. But then you wouldn’t be able to use environment variable and plan overrides. Instead, what you might do is hit a “plan” API endpoint that tells you what resources the render.yaml file would create. Then you’d manually apply the overrides to those resource definitions and deploy the resources one at a time. This is more work, but at the end of it you’d have total control over suspending, resuming, and deleting the resources that comprise your preview environment. The other challenge is that if there are dependencies between the resources defined in the render.yaml file, you’ll have to manage those yourself. (For example: wait to deploy a service until the database comes up, grab the database connection string and store it in an environment variable for the service.)

As I said, the API schema is a work in progress. We’ll keep your use cases in mind as we finalize it.

Being able to control each service individually is quite powerful indeed, but in my case I want to control my render.yaml as a whole when deploying/suspending/resuming/destroying it. I think this is the biggest value proposition that render provides today, the “declare the complexity once, deploy anytime, in any environment”.

I’d imagine higher constructs in the API that can take a render.yaml file, an optional preview flag to tell render to interpret the given render.yaml in a preview context, and then just the 4 commands to manipulate the whole group of services (deploy/suspend…).

According to the current workflow of the previews environments you already are able to deploy and destroy a render.yaml in a preview context as a whole. We just need to access this functionality to decide on the exact timing of these steps. :grinning_face_with_smiling_eyes: (but having suspend/resume on the whole render.yaml would also be awesome!)

Well said @jgoux. I’ve passed your feedback on to the rest of the API team.