Overview​

Cloud.gov is a Platform as a Service (PaaS). You deploy applications, while Cloud.gov operates and secures the underlying platform (hosts, containers, networking, orchestration).
Because customers do not have access to operating systems or container hosts in a PaaS model, OS-level endpoint agents cannot be installed. This design ensures consistency, security, and compliance across all tenants.

Customer Responsibility: Secure your application layer—this includes code, dependencies, data flows, logs/metrics, CI/CD security, and application incident response.

Shared Responsibility at a Glance​

Accessibility Note: The diagram shows that Cloud.gov manages infrastructure, operating systems, containers, networking, and platform monitoring. Customers manage application code, dependencies, logs, and testing. Endpoint agents are not permitted at the platform layer.

What You Can Do Instead​

Immediate Actions​

• Forward app logs/metrics to your SIEM or monitoring system using supported interfaces.
• Instrument CI/CD pipelines with static, dynamic, and interactive security testing (SAST/DAST/IAST).
• Generate and maintain a Software Bill of Materials (SBOM) for all builds; alert on changes to critical components.

Best Practices​

• Use in-process protections (e.g., runtime application self-protection libraries) within your app if compatible.
• Leverage threat intel and APIs for enrichment and detections at the application layer.
• Adopt provenance and artifact signing (e.g., SLSA-aligned pipelines) for supply-chain security.
• Develop and test an app-focused incident response plan that aligns with your agency’s processes.

FAQs​

Q: Why can’t I install an endpoint agent? Because PaaS boundaries prevent OS or kernel access. This is fundamental to Cloud.gov’s multi-tenant security model.

Q: Do we still get malware and threat monitoring? Yes—at the platform layer, operated by Cloud.gov. You are responsible for monitoring at the application layer.

Q: Could we run a sidecar container with an endpoint agent? If it requires host/privileged access or kernel hooks: No. If it operates purely at the app layer, evaluate it like any other dependency.

Q: Can we use EDR data without an agent? Yes. You can forward application logs, integrate APIs, and apply policies in your CI/CD pipelines.

Roles & Responsibilities​

• Cloud.gov (Inherited): Host OS hardening, container runtime, network controls, platform monitoring & incident response.
• Customer Responsibility: Application code/configuration, dependencies & SBOM, CI/CD security testing, app logs/metrics, and application-layer incident response.

Customer Responsibility: Ensure your System Security Plan (SSP) reflects inherited vs. shared vs. customer-owned controls accurately, using the FedRAMP Control Responsibility Matrix (CRM) as a guide.

Compliance Context​

Cloud.gov operates under a FedRAMP Moderate Authorization (package ID F1607067912). Customers inherit platform-level controls. You must implement and document application-level security controls in accordance with NIST SP 800-53 Rev 5 and the FedRAMP CRM.

If you have any additional questions, please contact support@cloud.gov and we would be happy to assist you.

The most important metrics for monitoring your application's health are the memory and CPU metrics, which can help you identify:

• If your application has enough memory provisioned per instance
• If your application is experiencing CPU spikes and associated performance issues
• If you need to horizontally scale your application instances

A note about CPU metrics​

Your application runs in one or more application containers distributed across Diego cell virtual machines (VMs). CPUs are virtualized and shared with other application containers on the VM.

The CPU usage figure reported as part of the application metrics represents the CPU usage of an application instance as a percentage of a single CPU core. Since there are usually multiple CPU cores per VM, it is possible for the CPU usage to exceed 100%, which means your application is using the equivalent compute power of more than a single CPU core.

A CPU usage figure above 100% is not necessarily problematic. The more important metric for identifying issues is CPU entitlement, which is a formula Cloud Foundry uses to determine how much CPU your application is allowed to use from the host VM based on its memory capacity.

If the CPU entitlement figure exceeds 100% for any application instance, then the instance is effectively borrowing spare CPU resources from the host VM. Since applications are regularly redistributed across the available host VMs, the amount of spare CPU capacity available on the VM can change, so any instances of CPU entitlement above 100% should be treated as an indication of insufficient application resources and addressed appropriately by adding more instances or allocating more memory.

While the CPU usage figure itself does not independently reveal application issues, it is still worth monitoring as a relative value, since sudden spikes in the value can still indicate abnormal performance of your application.

Retrieving current memory and CPU metrics​

To retrieve the current memory, CPU usage and CPU entitlement for your app, you can use the cf app command:

cf app APP-NAME

The output from running that command will look something like:

The cpu and memory metrics in the output can be interpreted as follows:

• cpu: percentage of CPU used by the application, as explained above.
• memory: memory used out of the amount of memory allowed for each application instance.
• cpu entitlement: CPU time used by an app instance as a percentage of its CPU entitlement.

How to view historical memory and CPU metrics in OpenSearch​

While it is useful to view your application's CPU and memory usage at a given point in time, what is usually more helpful is to see the trends in your application's health metrics over time and in particular whether they correlate to observed performance issues for your application.

Logs containing these application metrics are already ingested into the OpenSearch instance for customer logs. Furthermore, there are already built-in visualizations and dashboards for viewing these metrics for your applications.

To view the dashboards for application metrics, follow these steps:

1. Log in to OpenSearch.

2. Click Dashboard in the bookmark links at the top of the page.

3. Enter App - Metrics in the search bar on the Dashboards page and click the link to the App - Metrics dashboard in the results.

4. The App - Metrics dashboard should automatically populate with metrics for a graph of CPU usage, memory usage (in bytes), and disk usage for all of your applications. Add filters (e.g. @cf.space) to limit the results to the desired applications and adjust the time filter to see metrics for the desired time period.

If you encounter any difficulties using these dashboards or have any questions about them, please contact support@cloud.gov and we would be happy to assist you.

Dashboards is a user interface that lets you search and visualize your application logs. Dashboards has a user guide that explains more about how to use it and to create custom visualizations.

What are some types of logs?​

Cloud Foundry assigns a type to each log message depending on its origin. Application logs are assigned the APP log type. HTTP requests being routed to an app will produce the RTR log type. The various types of logs are listed in the CloudFoundry documentation.

The log type is stored on logs in the @source.type field. So to query for application logs, you could use a filter of @source.type: "APP".

Querying Logs​

Cloud Foundry logs have a few useful fields which will be helpful for querying logs. Cloud Foundry has 3 fields that can be very helpful to filter down what app logs show up.

• @cf.org - The organization, this is applicable to multi-org users
• @cf.space - The space in a organization.
• @cf.app - The app in a space.

Other fields that may be useful for querying:

• @message - the app specific message attached to the log. This field supports full-text searching, so you can do a search of @message: "foo" to find all logs where @message contains foo.
• @raw - the raw message received by OpenSearch before parsing into other fields. This field also supports full-text searching

How to visualize application traffic​

Router log data can be used to create a visualization of your application traffic following the steps below.

After you have logged into Dashboards, click "Discover" in the left sidebar menu. Then, add filters(under search bar) and search terms to query for router logs as seen in the screenshot below. Please note that the filters shown here for a specific space and application are just an example. You might want to view logs for all requests to application in a given space, in which case you would not want a filter for @cf.app.

The next step is to visualize your search results based on a specific field. To visualize request logs over time, choose the @timestamp field from the left sidebar of "Available fields". Then, click "Visualize".

By default, visualizing logs based on @timestamp will produce a histogram chart. To change the chart type to line, which might be more useful for this type of data, click the "Metrics & axes" link in the chart configuration panel on the right side of the screen. Then, under "Metrics" and "Count", select "Line" from the "Chart type" drop-down. Finally, click the "Update" button at the bottom right of the screen and the chart should update to a line chart.

Fields for router requests​

Listed below are the explanations of some field names for router (RTR) logs:

• rtr.app.id: The application guid
• rtr.hostname: The domain/hostname the request was sent to (e.g test.app.cloud.gov)
• rtr.http_user_agent: What user agent the request came from (Chrome, Firefox, Curl, etc…)
• rtr.path: The specific url path that was requested (e.g. /my/test/page)
• rtr.status: Gives the status of the request (200, 404, etc…)
• rtr.verb: The type of request (POST, GET, etc...)
• rtr.x_forwarded_for: The IP address the request came from
• rtr.timestamp: The time of the request in UTC

The full list of fields available for router logs can be found in our OpenSearch field mapping configuration

Generate report of logs​

OpenSearch allows for generating a report from a saved search. This is done by choosing a search and clicking the Reporting menu option.

Reports have a non-configurable 10,000 row limit. They have no explicit size limit (for example, MB), but extremely large documents could cause report generation to fail

The buildpack cache (aka "app cache," or "build artifacts cache") is a per-application cache that is stored in the blob (binary large object) store and recovered when an existing application is restaged (regardless of application code or buildpack changes).

Different buildpacks use this cache for different things. System buildpacks are cached on the VM hosting the application container and then mounted as read-only volumes into each staging container. If your application references a custom buildpack or buildpack version via a URLs (to a git repo or a zip file), those buildpacks are downloaded every time application staging occurs.

Potential cache issue with apt-buildpack​

The potential issue with the apt-buildpack cache involves the / directory disk allocation. The apt-buildpack uses /tmp for its cache like other buildpacks. In all buildpacks, the / directory is allocated approximately 9% of the disk allocation, meaning that the size of the /tmp subdirectory cannot exceed that disk allocation. For most buildpacks, this disk allocation is sufficient as all the possible packages that they could be installing is known with the only variable being the application code. However, the apt-buildpack can install an unknown number of packages of variable size, such as the AWS CLI. Depending on how many packages are installed and their individual size, the allotted 9% of disk space may not be sufficient, causing application staging to fail with an error like:

stdout: No space left on device. /usr/bin/tar: /tmp/output-cache: Wrote only 2048 of 10240 bytes

How to address the issue​

If the number of packages that you are installing during staging is within the allotted 9% of the disk limit (2 GB default) you can attempt to resolve the issue by clearing the buildpack cache and attempting to push your application again. Please refer to the commands below to clear your buildpack cache.

app_guid=$(cf app --guid <staging-app-name>)
cf curl -X POST /v3/apps/${app_guid}/actions/clear_buildpack_cache

You can additionally configure the apt buildpack cache to purge any cached content by setting cleancache to true which calls apt-get clean and apt-get autoclean and is useful to remove any cached content per the apt buildpack documentation.

If you encounter any difficulties deploying this solution or have any questions, please contact support@cloud.gov and we would be happy to assist you.

As we announced on November 21, 2024, we are upgrading the Cloud.gov customer application logging system and the user interface at https://logs.fr.cloud.gov. The application logs interface prior to December 2024 was based on Kibana, and we are migrating to one based on OpenSearch Dashboards.

While all the underlying functionality is unchanged, or improved, there are some differences between Kibana (old) and OpenSearch (new), which we'll outline here.

Note: You will need to migrate your saved objects (searches, visualizations) from Kibana to OpenSearch before January 7, 2025. After that date, importing objects will require a support request.

Logging in to the new system​

The first time you log in to the OpenSearch-based system you'll be presented with the following OpenId "Application Authorization" dialog:

You'll need to accept all the scopes. If for some reason you need to revoke access later, you can do so at: https://login.fr.cloud.gov/profile

You'll then need to choose the Cloud.gov org you want to work with in the "Select you Tenant" dialog:

If you have access to multiple orgs, you can switch your tenant later by clicking the OpenSearch user avatar on top right.

The main OpenSearch Dashboard should resemble the Kibana dashboard. If you're provided a Dashboard selection screen (see below), choose "App - Overview".

Otherwise the main navigation menus should be familiar and you're now ready to explore your Cloud.gov logs with OpenSearch.

Migrating Kibana customizations to OpenSearch​

All of the application and CloudFoundry logs that have been available in Kibana will be available to you in OpenSearch. You needn't take any action to ensure that. You will need to migrate custom dashboards and saved searches by exporting them from Kibana and importing them into OpenSearch.

Export Saved Objects from Kibana​

In Kibana, use the left navigation menu to select "Management" -> "Stack Management":

Then in the Stack Management view, select "Saved Objects" under the "Kibana" heading:

From the Saved Objects screen, you can search for the visualizations or searches you've previously saved. From that screen you can export all the matching objects as a single export.ndjson file (as shown below), or as individual .ndjson files:

Importing saved objects into OpenSearch​

If you used the same saved object in Kibana across multiple Cloud.gov orgs, you will need to import it into each OpenSearch tenant (each tenant corresponds to a Cloud.gov Cloud Foundry "org").

Once you've exported the objects as .ndjson files, switch to OpenSearch, then:

• From the LeftNav menu, select, "Management -> Dashboards Management".
• On the "Dashboards Management" window, select "Saved Objects" on the left menu.
• Then select "Import" on the upper right corner.
• On the "Import saved objects" window:
  • Select the file to upload
  • For "Import options" select:
    • "Check for existing objects"
    • "Request action on conflict"
  • The click the "Import" button, as show below:
  • If the import results in an "Overwrite index-pattern" dialog, you will likely want to "Skip" the overwrite:
  • When the import is complete, click "Done"

Recovering Saved Searches and Visualizations after Kibana decomissioning​

If you missed migrating a Saved Object from Kibana to OpenSearch, and Kibana is no longer available, please contact support@cloud.gov. We have saved all customer objects and can recover those for you.

User interface changes​

The screenshot below show some of the major changes to the user interfaces, such as:

1. The drop down menus have moved from the upper left to the upper right.
2. The "Top 5 values" for a field view is now an option to the right of the field, instead of a double-click.
3. There are a lot more values gathered for container metrics (see below) so you may notice a higher message count in OpenSearch.

Key system differences​

The Cloud.gov team has implemented OpenSearch to deliver a number of benefits to our customers. Among these are:

• Twelve months of live access to system logs, in alignment with M-21-31.
• Definitions of saved searches and visualizations are now isolated by OpenSearch tenants that correspond to Cloud.gov organizations.
  • You no longer need to worry about choosing a globally unique name.
  • If you share the same saved object across multiple orgs, you will need to import it into each of your orgs.
• Better handling of large log messages. Both Kibana/ELK and OpenSearch have a 32kb limit on message size. The older system dropped such messages from Kibana (although they were still retained in cold storage), the newer system, for JSON messages, keeps the first 32kb and discards the rest
  • Truncated messages are tagged with _messagetrimmed.
  • Extremely large log messages (over 1Mb) are trimmed and tagged _logtrimmed -- such message are probably indicative of a coding error in your application.
  • You can search for such messages with a filter of @logs is one of "_messagetrimmed", "_logtrimmed", as shown here
• AWS Brokered Service Logs (Beta): If your Cloud.gov organization had already worked with Cloud.gov support to enable publishing of your RDS database logs to Cloudwatch, then you can search for those logs with the filter @version: 1. Most databases, as of December 2024, are not yet publishing their logs to CloudWatch and thus will not appear in OpenSearch.
  • Cloud.gov will be expanding the availability of logs from other brokered services in 2025. This is a beta feature and subject to change.
  • Cloud.gov is working to make enabling publishing of RDS database logs to CloudWatch a self-service feature of the database broker, but there is no ETA yet for delivery of this feature.
• JSON log parsing: Custom logs are not at risk of being dropped because of index field limits. JSON logs are now ingested using the flat_object field type in OpenSearch. The flat_object field type allows for searching nested fields of a JSON object using dot notation.
• Additional container metrics: We now log additional container metrics, available under the containermetric.name field. Particularly useful is the containermetric.name: cpu_entitlement, which is a way to track whether you're exceeding the allowed CPU for your apps.

Reporting Issues and Getting Help​

Report any undocumented issues you encounter, or questions you may have, to support@cloud.gov.

The reason why this issue is occurring is because the recently updated buildpack versions are trying to clone some git submodules over SSH instead of HTTPS. Cloud.gov supports HTTPS egress for buildpacks during staging, not SSH, as such the buildpack is unable to be fetched and the staging process fails. An issue with the PHP buildpack has already been posted to the CloudFoundry GitHub.

The solution to this issue is pin the version of the buildpack that you are using with your application to version 4.6.23 or a prior version.

To pin the buildpack version, specify the buildpack version URL in your application manifest under the buildpacks: attribute as shown below.

buildpacks:
  - https://github.com/cloudfoundry/php-buildpack#v4.6.23

If you encounter any difficulties deploying this solution or have any questions, please contact support@cloud.gov and we will be happy to assist you.

Public vs Private repositories​

A repository does not need to be public in order for Pages to be able to build the website and there are no added website security benefits for private repositories; both public and private repositories are acceptable on Pages due to all branches, both public and private, being published publicly when built. When it comes to deciding whether to make your GitHub repository public or private you may consider the following items.

Public​

Pros​

• Easy for Pages engineers to access the codebase and provide support/debug
• Transparency/High visibility
• Able to utilize CodeQL free of charge
• Seamless OpenSSF Allstar integration
• Collaboration with outside contributors
• Compliance with open source and transparency initiatives per agency/program

Cons​

• Sensitive information inadvertently hardcoded is immediately visible

Private​

Pros​

• Test/Make changes to a website without exposing commits to the public
• Safeguard sensitive data such as API keys, access tokens or other credentials by using environment variables in the build runtime. Websites on Pages are all published publicly, user's configuration settings and static site build engine factor into what is included in the site build output.

Cons​

• Not able to utilize CodeQL for free
• Potentially out of compliance with agency/program transparency initiatives

Repository User Permissions​

It is important to designate admin/owner access of the repository to one or more individuals who are either the persistent site owners or else federal employees with the “manager” role within the Pages organization. It is strongly recommended to not grant admin/owner privileges of the repository to any contractors or other temporary employees. This is because in certain instances where actions will need to be taken by either Pages engineers or by customers that require an active user with admin access to a website's repository. Additionally, when adding contributors or collaborators to a private repository it should mirror the member count that your organization has within Pages. Please note that removing an organization member from an organization within Pages will not automatically remove them from the repository within GitHub. They will still retain whatever read/write access they have in GitHub until explicitly removed.

GitHub Repository Security​

Dependabot​

Dependabot is a useful and highly configurable automated dependency management tool which assists developers with keeping third-party dependencies up-to-date with their latest version. This can benefit smaller developer teams by reducing maintenance overhead. Dependabot scans the repository for outdated dependencies and generates pull requests against whatever branch it is enabled on to update them. Check out our knowledge base article on Dependabot to learn how to enable and configure it for your repository.

Branch Protection​

Branch protection is a security feature within GitHub that limits users with write and push access the ability to push directly to any branch where branch protections are enabled. This includes but is not limited to requiring a pull request to merge code, setting the number of pull request reviews required, requiring status checks, dismissing approvals and other configuration options. With good branch protection settings you can protect your repository from an attack by a malicious actor with the stolen credentials from a single maintainer of that repository.

OpenSSF Allstar​

Allstar is an open source security tool which monitors organizations and repositories within GitHub for adherence to security best practices and improves the security posture for those organizations. It achieves this by continuously scanning repositories for any breach of enabled security policies and then raising an alert for any security remediation findings. This alert can be in the form of a GitHub issue or another type of configured action. In some instances Allstar can automatically resolve the issue on the admin/owner's behalf.

Additional Advanced Feature Settings for GitHub Repository Security​

CodeQL​

CodeQL, a GitHub open source security application, is a code analysis tool that automatically finds security vulnerabilities and other software bugs in codebases. Although primarily used for security analysis it can also be used for general code quality analysis as well. It is important to again note that CodeQL is only free to use on public repositories and GitHub Advanced Security must be purchased/enabled in order to utilize the tool on private repositories.

Signed Commits​

In conjunction with good user permissions practices and as an added layer of security you can ensure that only commits by members of your Pages organization are made to the repository by having them signed and then verified by GitHub. This increases the authenticity and integrity of your repositories' commit history, especially for those repositories which are public facing. There are various methods when it comes to implementing commit signing so be sure to review them in full detail.

The most important metrics for monitoring your application's health are the memory and CPU metrics, which can help you identify:

• If your application has enough memory provisioned per instance
• If your application is experiencing CPU spikes and associated performance issues
• If you need to horizontally scale your application instances

A note about CPU metrics​

CPUs are virtualized and shared across application containers on a Diego cell virtual machine (VM).

The CPU usage figure reported as part of the application metrics represents the CPU usage of an application instance as a percentage of a single CPU core. Since there are usually multiple CPU cores per VM, it is possible for the CPU usage to exceed 100%, which means your application is using the equivalent compute power of more than a single CPU core.

A CPU usage figure above 100% is not necessarily problematic. The more important metric for identifying issues is CPU entitlement, which is a formula Cloud Foundry uses to determine how much CPU your application is allowed to use from the host VM based on its memory capacity.

If the CPU entitlement figure exceeds 100% for any application instance, then the instance is effectively borrowing spare CPU resources from the host VM. Since applications are regularly redistributed across the available host VMs, the amount of spare CPU capacity available on the VM can change, so any instances of CPU entitlement above 100% should be treated as an indication of insufficient application resources and addressed appropriately.

Cloud Foundry provides a cf CLI plugin for determining if any of your application instances are exceeding their CPU entitlement.

While the CPU usage figure itself does not independently reveal application issues, it is still worth monitoring as a relative value, since sudden spikes in the value can still indicate abnormal performance of your application.

Retrieving current memory and CPU metrics​

To retrieve the current memory and CPU usage for your app, you can use the cf app command:

cf app APP-NAME

The output from running that command will look something like:

The cpu and memory metrics in the output can be interpreted as follows:

• cpu: percentage of CPU used by the application, as explained above
• memory: memory used out of the amount of memory allowed for each application instance

How to view historical memory and CPU metrics in Kibana​

While it is useful to view your application's CPU and memory usage at a given point in time, what is usually more helpful is to see the trends in your application's health metrics over time and in particular whether they correlate to observed performance issues for your application.

Logs containining these application metrics are already ingested into the Elasticsearch/Kibana instance for customer logs. Furthermore, there are already built-in visualizations and dashboards for viewing these metrics for your applications.

To view the dashboards for application metrics, follow these steps:

1. Log in to Kibana.

2. Click Dashboard in the bookmark links at the top of the page.

   

3. Enter App - Metrics in the search bar on the Dashboards page and click the link to the App - Metrics dashboard in the results.

   

4. The App - Metrics dashboard should automatically populate with metrics for a graph of CPU usage, memory usage (in bytes), and disk usage for all of your applications. Add filters (e.g. @cf.space) to limit the results to the desired applications and adjust the time filter to see metrics for the desired time period.

   

5. If you want to see a dashboard of the same application metrics, but grouped by each application instance instead of just each application, repeat steps 3 - 4 and search for a dashboard name App - Metrics by instance.

   

If you encounter any difficulties using these dashboards or have any questions about them, please contact support@cloud.gov and we would be happy to assist you.

Taking advantage of end-to-end Cloud.gov services gives our agency customers baked-in security to help ease their compliance burden, and we provide support for dual agreements to make accessing these services simple. Below, we share how to display dynamic content using a lightweight database and API on Cloud.gov connected to a Pages static website. Once you see how easy it is, we hope you’ll consider situations where your static Pages sites could be enhanced with dynamic content using additional Cloud.gov services.

What you’ll need to get started​

1. Some data you need to store, process, or access separately from the static site. In this case, we pulled a CSV file dataset from the FDIC, a Pages customer, which is hosted on data.gov, another office alongside Cloud.gov in TTS.

2. A database and environment to store the data, provided by Cloud.gov. In our example, we chose an RDS instance(Relational Database Service) of the PostgreSQL v15 database due to its rich feature set and simple methods of storing data from a CSV.

3. A server-side application that securely accesses the database and responds to HTTP requests. Our example uses a simple API flask application deployed on Cloud.gov

4. The static website hosted on Cloud.gov Pages. Our example uses a simple HTML file, but you could use any static site generator or single-page application on Pages.

How it all comes together​

The below diagram shows the relationships between the three services:

1. The PostgreSQL database, which houses the dataset and returns the results found by the query. The fetched data is sent back to the API application, and the API application passes it on to the static site in the response body as JSON.

2. The API application first securely connects to the database by retrieving the RDS service credentials via Cloud.gov’s Cloud Foundry environment. Once it establishes a secure connection to the PostgreSQL database, the application executes a SQL query against the database.

3. The static website, hosted on Pages, makes an HTTP fetch request for some dynamic content to the Cloud.gov-hosted API application.

Set up a database using Cloud.gov’s RDS​

Before you can create an API to access the data, the data must be available in the appropriate Cloud.gov org and space. It’s relatively straightforward to provision a database of your choice and make it available for querying from other applications using Cloud.gov’s RDS.

When you create an RDS service instance, Cloud.gov automatically creates the default username and password for the database. If you want to allow an application (like our Flask API app) to access that database, you can bind that RDS database service to your application to safely expose those credentials as environment variables within the Cloud.gov environment. This means your Cloud.gov API application will have privileges to data that your Pages app will not, which can be useful in separating data concerns and keeping some information private.

In our case, we provisioned a PostgreSQL v15 database instance within Cloud.gov to store some sample data. For the purposes of filling out an example database, we sourced a CSV from FDIC which itemizes recently failed banks, and then imported the CSV into the database’s single row table using the syntax COPY/FROM.

When queried via the API application, the database will select a number of rows, convert them to a Python dictionary via the Row factories module, and return that collection.

Bank Name                | City          | State | Cert  | Acquiring Institution               | Closing Date | Fund  | id
-------------------------+---------------+-------+-------+-------------------------------------+--------------+-------+---
Citizens Bank            | Sac City      | IA    | 8758  | Iowa Trust & Savings Bank           | 3-Nov-23     | 10545 | 1
Heartland Tri-State Bank | Elkhart       | KS    | 25851 | Dream First Bank, N.A.              | 28-Jul-23    | 10544 | 2
First Republic Bank      | San Francisco | CA    | 59017 | JPMorgan Chase Bank, N.A.           | 1-May-23     | 10543 | 3
Signature Bank           | New York      | NY    | 57053 | Flagstar Bank, N.A.                 | 12-Mar-23    | 10540 | 4
Silicon Valley Bank      | Santa Clara   | CA    | 24735 | First–Citizens Bank & Trust Company | 10-Mar-23    | 10539 | 5

If you’re thinking of trying this yourself, know that you aren’t limited to using RDS and PostgreSQL for storing the data; you have your choice of database infrastructure hosted on Cloud.gov. Check out more options for database services provided by Cloud.gov in our CloudFoundry marketplace. Please note that database options are limited for those working in a Cloud.gov sandbox.

Set up a server API Application in your Cloud.gov org space​

You may write the server-side API in whatever language you’re comfortable with; we chose a simple Flask application in Python. Our example app which is hosted on Cloud.gov serves as the interface between the PostgreSQL database and the Pages website. Through the Flask application’s endpoint, the Pages website can request data and receive structured responses via HTTP. In our example app, the Flask app fetches the 15 most recent results from the PostgreSQL database table and returns them via HTTP as an JSON response. JSON is a reliable choice for easy parsing from the front-end static site using JavaScript, but you could also send XML, a string, or another data type.

1. Import the modules​

In order for our example web server to handle HTTP requests, dispatch responses, and interact with the database and environment, we needed a handful of Python modules:

- flask for creating the web server and preparing the JSON response

- os for reading environment variables and returning the result

- cfenv for interacting with Cloud Foundry environment variables

- dict_row for fetching database results as dictionaries

- flask_cors and CORS for handling Cross-Origin Resource Sharing (CORS), which is what allows Pages and the server to talk to one another even though they’re on separate domains

- ConnectionPool for managing the set of open PostgreSQL database connections, this allows for efficient reuse of existing open connections which reduces latency

- OperationalError for reporting database connection failures, if a failure occurs the application will be able to recognize it and automatically recreate the connection pool

- atexit ensures proper cleanup of database connection pool upon application termination

2. Set up the secure database connection​

Because we’ve set up the database using RDS and bound to the service in this Cloud.gov org space, we have direct access to the database credentials via application environment variables. Cloud.gov and RDS make it easy to set up a secure connection to the database using these environment variables. Here is how we connect to our database using the DATABASE_URL environment variable provided by the Cloud.gov platform in our Flask application:

aws_rds = app_env.get_service(name="example-website-api-database")

def setup_connection_pool():

    database_url = os.getenv("DATABASE_URL")

    pool = ConnectionPool(
        conninfo=database_url,
        min_size=1,
        max_size=20,
    )
    return pool

3. Define the routes​

The server application can now access the database, so we need it to provide endpoints for our static site to actually make the requests. These endpoints are defined in the Flask application’s routes. Our example app has two endpoints that can handle GET requests.

• The root-level route ("/") is accessed when an HTTP request is made to the bare domain itself; in our case, it will react to requests of https://cfpyapi.app.cloud.gov/ with a simple string response letting you know the app is running. It’s a best practice not to use your root-level domain for serving data anything other than some general information about your API service.
• The second route ("/get_table") is the one we use for accessing data from the database. When an HTTP request visits https://cfpyapi.app.cloud.gov/get_table, the Flask app will execute our get_table() function, which queries the database using the above credentials, finds matching results, and returns them in JSON format. Serving data and handling requests from a route endpoint makes the API more organized and easier to understand. When you need more endpoints for other queries in the future, it’s easy to add more routes.

@app.route("/", methods=["GET"])
def hello():
    try:
        return "There is a table right behind this door!"


@app.route("/get_table", methods=["GET"])
def get_table():
    global connection_pool
    try:
        return fetch_fdic_banks(connection_pool)

4. Make the API available to your Pages app​

By default, a Flask web server like ours handles HTTP requests, executes the queries, and sends back JSON responses to the client, as long as the request comes from the same origin (or domain). The initial CORS settings assume you would not want websites and applications from other domains to be able to make requests of yours. In order for the app to respond to a request from a static site, that request origin must be specifically allowed in the CORS settings. It’s good security practice to use environment variables whenever you’re able to so we’ve stored our Pages website URL as an environment variable in the application instance using cf push --var ORIGIN='<site>.sites.pages.cloud.gov'.This not only enables us to avoid having to hardcode the full URL but also if we had different environments ie. development, staging and production we can easily change the origin without having to modify the code. If your Pages site uses a custom domain, that would be the origin to use.

origin = os.getenv("ORIGIN")
port = int(os.getenv("PORT", 8080))
app = Flask(__name__)
CORS(app, origins=origin)

Once your server API can connect to the database, execute queries at specific routes, and respond to only your Pages site domain, you’re ready to deploy. For instructions on how to deploy applications on Cloud.gov, please refer to the official Cloud.gov documentation.

Serve content to the Pages website​

Now, you’re ready to request dynamic content from your static site. In our example, the Pages-hosted site is a single index.html file with some CSS and image assets and a few lines of JavaScript to make the request and display the dynamic content. We’re using an already minified version of the U.S. Web Design System (USWDS) library from a Content Delivery Network (CDN) so there are no build tasks, transpiling, preprocessing, or compiling in this project – just a simple static site with some nav and the basic structure of a table, waiting to be filled with dynamic content.

There are two steps left: Making the HTTP request to the server app, and then displaying the response that the server app provides. Both are handled with straightforward JavaScript.

1. Make the request​

First, our static site makes an asynchronous request to the API. There are a few ways to do this; our example function getDataFromCloudAPI() uses the fetch pattern to make a request at the /get_table endpoint, waits for the response using await, and then parses it into JSON when the Promise resolves. We’ve wrapped the request in a try/catch block to log any errors to the console in the browser if the request fails. Once resolved, the function ultimately returns the data requested in JSON.

async function getDataFromCloudAPI() {
  let dataJson = null;
  try {
    const response = await fetch("https://cfpyapi.app.cloud.gov/get_table");
    dataJson = await response.json();
  } catch (error) {
    console.log(error);
  } finally {
    return dataJson;
  }
}

2. Display the dynamic content​

To keep our example simple, our static site displays the API-provided data in a simple HTML table. Using the JSON response data, we first build the table’s headers and then the rows, one by one, and append them all to an empty HTML table element that already exists in the static page. Using the source data to build both the table headers and the content rows means they will always match, even if the data structure on the server or database changes.

function createTableHead(rowData) {
  const tableHead = document.createElement("thead");
  let newRow = tableHead.insertRow();

  Object.entries(rowData[0]).forEach(([key, value], cellIndex) => {
    if (key !== "id") {
      let newHeader = document.createElement("th");
      newHeader.innerHTML = key;
      newRow.appendChild(newHeader);
    }
  });
  return tableHead;
}

function createTableBody(rowData) {
  const tableBody = document.createElement("tbody");
  rowData.forEach((row, rowIndex) => {
    let newRow = tableBody.insertRow(rowIndex);
    Object.entries(row).forEach(([key, value], cellIndex) => {
      if (key !== "id") {
        let newCell = newRow.insertCell(cellIndex);
        newCell.innerHTML = value;
      }
    });
  });
  return tableBody;
}

function createTable(data) {
  document.getElementById("data-table").appendChild(createTableHead(data));
  document.getElementById("data-table").appendChild(createTableBody(data));
}

And that’s it! The static HTML page makes the fetch request to the API on load, then builds the table contents using the response, seamlessly.

Serving dynamic content from a backend database via an API to a static site hosted on Cloud.gov Pages is easy within the Cloud.gov ecosystem. We proudly offer a suite of secure and compliant databases which you can leverage to store data and dynamically display within their Pages websites without the need to manually update the static files or struggle with large data collections checked into the static site repo.

If you’re interested in enhancing your static sites with dynamic content, we’d love to help you set up a dual agreement for Cloud.gov and Pages services with your agency or office. If you’re already a Pages customer, it takes a simple modification to your current IAA to access services from Cloud.gov to get started. Launching an app on Cloud.gov will require an Authorized, and we’re here to help you through every stage. For more information, please reach out to inquiries@cloud.gov.

For support with implementation and general questions about Pages sites, please reach out to pages-support@cloud.gov.

Government web applications and services are targeted by a barrage of ever-increasing challenges, probes, and malicious traffic. These attacks try to exploit common vulnerabilities, compromise infrastructure, or disrupt services for legitimate users. With thousands of customer application instances running on Cloud.​gov, we see many different kinds of attacks. Our commitments to safety, security, and transparency ensure that our defenses are up-to-date and robust, safeguarding Cloud.​gov customer applications from the impact of high-traffic-volume attacks directed at both the platform and our agency customers.

One of Cloud.gov’s priorities is to make sure that agencies can focus on their core mission without the burden of dealing with the complex and changing discipline of web application security. We ensure that all customer apps on Cloud.gov are protected by a strategic collection of web application security best practices developed to combat the kinds of threats that government websites are uniquely prone to experience. These strategies are part of our more complete security posture, which is continually emerging and evolving in response to the malicious actors and the attacks they attempt.

cloud.gov’s Web Application Firewall (WAF) stands as one protective barrier against various cyber attacks, operating between a web application and the internet. It acts as a vigilant gatekeeper, monitoring, filtering, and blocking malicious HTTP traffic, while allowing legitimate traffic to access appropriate resources. A specific set of WAF rules define the criteria for allowing or blocking incoming requests to all Cloud.​gov managed websites and applications.

Challenge Requests and Rate Limiting​

In order to make sure that an HTTP request to access a given web application is a legitimate visit and not a bot, Cloud.gov issues a silent browser challenge. It’s similar to a CAPTCHA, but doesn’t require the visitor to successfully solve a puzzle, as it runs in the background of the browser. This approach helps implement effective rate limiting, managing spikes in requests during high-traffic events, and separating legitimate traffic from probing or DDOS-style attacks. Cloud.gov’s WAF rules also make sure that a single IP address can pass no more than 2,000 of these challenges within a 5-minute period, so that even as malicious attacks become more convincing, customer applications on Cloud.gov remain available and responsive.

After implementing these WAF rules, we monitored traffic surges to ensure that the change handled situations resembling these types of attacks without any negative impact to Cloud.gov customers. For example, consider this graph of traffic across the entire platform from November 7th, 2023:

Reputation Lists​

Another element in Cloud.gov's WAF strategy is the use of existing reputation lists to exclude known potentially malicious or automated traffic that originates from specific IP addresses. At times, traffic surges to customer applications come from IP addresses that were previously determined to be bots or other threats. Cloud.gov protects all of our customer applications by blocking traffic from these known, low-reputation IP addresses. We monitored a potential attack from known low-reputation IP addresses on the evening of November 3rd, and despite seeing more than one million requests in a span of 15 minutes from these IP addresses, the reputation list WAF rule blocked the vast majority of the potentially malicious traffic:

Just as the Cloud.gov web application security approach guards against known threats using adaptive reputation lists, the Cloud.gov team flexibly and skillfully reacts to new situations. Real-world incidents, like a surge in potentially malicious traffic experienced last October, serve as valuable lessons for how cyber threats are changing for government web applications. Our team not only responds quickly during these incidents but actively works on proactive measures to prevent future occurrences. This iterative process of learning and improvement ensures that our security measures adapt to emerging challenges. We publish updates like these to reflect our total commitment to providing a resilient and reliable platform for our users.

Operating as a public entity in the government space means that malicious actors will regularly attempt to disrupt the services being provided to customers and stakeholders or to compromise the underlying infrastructure. At Cloud.gov, we strive to make sure agencies can trust that their applications are in capable hands, allowing their teams to focus on innovation and development while we handle the rest.

If your development team is interested in hearing more about Cloud.​gov, scheduling a demo, or getting access to a free sandbox, email us at support@cloud.gov.

In this example we will be taking a look at a simple single-page application which uses the React library and React Router v6. You can view a repository with a full example here and see the results in Pages here. There are also more detailed instructions on how to run this locally in the repositories’ README.md. This article serves as a high level overview and will highlight three key features:

• Application routing
• Environment variables
• 404 Pages

Application Routing​

Static site generators like Jekyll and Hugo handle routing by creating an HTML file at each path that can be requested by the user. For single-page applications, we will only render one HTML file: index.html, which loads the JavaScript necessary for running our application and router.

All of the routing is located in the main.jsx file. Here we import all our components, CSS and functions. We create the <BrowserRouter> and use the <RouterProvider> to provide that router to our application. The parent route registers a Layout component which essentially wraps all the other page components thus providing a uniform structure/design for all the components.

const router = createBrowserRouter(
  createRoutesFromElements(
    // set route as the path plus layout
    <Route path="/" element={<Layout />}>
      <Route index element={<Home />} />
      <Route path="stuff" element={<Stuff />} />
      <Route path="contact" element={<Contact />} />
      <Route path="*" element={<Page404 />} />
    </Route>
  ),
  { basename: import.meta.env.BASE_URL }
);

createRoot(document.getElementById("root")).render(
  <React.StrictMode>
    <RouterProvider router={router} />
  </React.StrictMode>
);

Additionally within the Layout component is where we house all the NavLinks and navbar which are located at the top of the page. The Outlet component tells the react router where to output the child route page components within the layout.

class Layout extends Component {
  render() {
    return (
      <div>
        <h1>Simple SPA</h1>
        <ul className="header">
          <li>
            <NavLink to="/">Home</NavLink>
          </li>
          <li>
            <NavLink to="stuff">Stuff</NavLink>
          </li>
          <li>
            <NavLink to="contact">Contact</NavLink>
          </li>
        </ul>
        <div className="content">
          <Outlet />
        </div>
      </div>
    );
  }
}

It’s important to remember that none of these routes are creating new HTML files at a given path, they are only rendering in the browser (unless you prerender with react-snap). We’ll see how to handle situations when the URL doesn’t match the HTML file in a later section.

Environment variables​

Pages provides certain environment variables in each build container and you can also add customer variables in your “Site Settings”. In the case of vite, the library we’re using in this example, we are using a single variable provided by Pages in two separate places: - BASEURL: This variable is supplied by the Pages build container and cannot be modified. This will insert the Pages site path based on the branch and associated domain. We’ll utilize this environment variable in our config file, vite.config.js, to set the base variable which will be used to update our assets path.

base: process.env.BASEURL;

Once the base variable is set in the vite configuration, it statically injects a variable, import.meta.env.BASE_URL, at build time. In our main.jsx file react router can use that variable to set opts.basename which ensures that routing is handled correctly regardless of where the application is served from. When a site is viewed at a non-production domain or preview URL it is not served from the root of the domain but rather a sub path.

const router = createBrowserRouter(
   …,
  {basename: import.meta.env.BASE_URL}
);

404 Pages​

In our application code, we created a 404 page as another component to be imported into the index.jsx file to be handled by react-router. The Route for the 404 page was set as a wildcard catch-all so that when any path entered is not /, /contact or /stuff the Page404 component will be rendered.

<Route path="*" element={<Page404 />} />

If users navigate to a sub-path of your site directly, we need special handling for this type of 404 because the application will not have loaded your index.jsx file (or associated router).

It is important to note that:

1. Specifically for any single-page applications hosted on Pages you will need to reach out to an Pages operator to manually set your custom 404 behavior to point to index.html.

2. This will only work on your production URL, and only when you have a custom domain. For preview links, you’ll need to navigate to the base URL first before moving on to other paths.

One of the benefits of Cloud.gov is the ability to deploy applications with a simple command.

By default, applications are deployed with a single instance which handles all traffic and load for your application. The downside of a single application instance is that if you have unexpected surges in application load, it is likely that your instance may run out of available CPU or memory or both, leading to an outage for your application.

To increase your application's ability to respond to requests, also known as availability, you can horizontally scale your application by running multiple application instances. When you have multiple application instances, your application requests are load-balanced among them to ensure that no single instance is prematurely overloaded, thus maximizing your availability.

By default, the routing infrastructure in Cloud.gov distributes requests to application instances using a round-robin algorithm.

Running multiple application instances also increases the chances that your application will be balanced across availability zones.

How to scale your application instances​

Using the cf push command, you can use the -i flag to indicate the number of application instances you would like.

For example, pushing an application with 2 instances:

cf push myapp -i 2

Additionally, you can also define the number of application instances in your application manifest.yml file with the instances manifest attribute.

An example for defining an application with 2 instances in your manifest.yml file:

    memory: 512mb
    instances: 2
    applications:
        name: myapp
        path: .

Please note the default number of instances is 1 instance.

You can also use the cf scale command to increase the number of instances for a running app.

cf scale myapp -i 2

Please note that running multiple instances may sometimes cause scheduled tasks or data loads to run multiple times. This issue can be prevented by updating scheduled tasks to use the CF_INSTANCE_INDEX environment variable, which denotes a specific application instance number.

Application instances and memory usage​

Each individual application instance utilizes the same amount of memory that is specified in the application manifest or indicated in the cf push command. Please note that the application cannot use more than the defined memory quota for your org.

For example, if the org my-example-org had:

• a memory quota of 3 GB and were hosting a single application myapp
• 256 MB of memory per application instance
• 4 application instances

Then, the application would be utilizing 1 GB (256 MB * 4 instances) of the org’s 3 GB total memory quota. This would leave 2 GB available for the org to otherwise use.

Dependabot is a feature of GitHub whose main purpose is to assist developers in staying on top of their dependency ecosystem. It does this by automating the dependency update process which in turn proactively addresses any potential security concerns. Dependabot also explicitly raises pull requests to address security vulnerabilities in dependencies and alerts developers to the precise location in the code where the vulnerabilities are. When enabled, it achieves this by scanning a project's dependency files such as package manifests (ie. package.json, Gemfile) or configuration files to identify dependencies and their versions. It then compares the current versions against the latest versions in the relevant package registries. As a developer woking on a site hosted on Pages you may have seen many of these types of alerts/pull requests in your given repository. This article will explain on how to effectively manage these alerts without compromising your code's security.

There are 3 distinct functions of Dependabot.

• Dependabot alerts: Alerts regarding dependency vulnerabilities where pull requests are automatically opened.

• Dependabot security updates: Additional alerts/information regarding vulnerabilities in any out-of-date dependencies where a vulnerability alert (non mergeable pull request) is opened in relation to the initial out of date dependency alert. If applicable it is linked to the actual dependency pull request.

• Dependabot version updates: Automatic pull requests opened when version bumps are available for dependencies.

* For Pages users with existing jekyll sites Dependabot alerts was automatically enabled upon launch. This is not the case for users who have 11ty or Gatsby based sites where by default all Dependabot services are disabled upon launch.

Where to locate and turn on dependabot​

To enable any of the Dependabot features navigate to the rightmost tab “settings” in the GitHub repository. On the left hand panel under security select “Code security and analysis”. Here you can enable either of the three previously discussed settings. You can enable Dependabot alerts and Dependabot version updates singularly as they are mutually exclusive of the feature. Enabling Dependabot security updates will also enable Dependabot alerts as it is reliant on alerts to function.

Where to locate and configure the dependabot.yml file​

The best way to take advantage of Dependabot while not getting overwhelmed by the amount of pull requests opened is to override the default behavior by customizing the dependabot.yml file which is located in the .github folder of the repository. You can either create it manually or have it automatically generated when enabling Dependabot version updates.

There are certain configuration options which can be set in your dependabot.yml file which will help reduce noise and cut down the amount of open pull requests in your repository. The options that we will focus on specifically are as follows:

interval: Set the frequency that Dependabot will check for new versions for each defined package manager.

open-pull-requests-limit: Modify the max limit of pull requests opened for version updates.

ignore: Explicitly ignore specific dependencies, versions and update-types.

target-branch: Specify a different branch for manifest files and for pull requests instead of the default branch.

For a complete list of configuration options and implementations, visit the GitHub site via Configuration options for the dependabot.yml file.

This example dependabot.yml file is for any Jekyll site running on Pages. In this case there are two defined package ecosystems which are npm and bundler, both located in the root directory. Specifically for the npm package we’ve set the conditions for Dependabot to check for dependency updates once a week, on Monday, while specifying the time and timezone. By default, Dependabot is set to the UTC timezone. We’ve added and set the condition open-pull-requests-limit to the value 2 so no matter how many updates there are, Dependabot will only open 2 pull requests a week maximum which will drastically cut down on noise. By default, Dependabot opens a maximum of five pull requests for version updates. We have also set an ignore condition for the dependency minimatch which will ignore all updates through version 3, effectively pinning it at its current version for the time being. For the bundler package manager we have kept the same conditions as with npm except we have set the frequency interval to monthly which by default will be the first Monday of the month. Although this is in the same yaml file, conditions set for each package-ecosystem are independent of each other. We have also specified a bundler dependency version to ignore.

version: 2
updates:
  - package-ecosystem: "npm"
    directory: "/"
    schedule:
        internval: "weekly"
        day: "monday"
        time: "07:57"
        timezone: "America/New_York"
    open-pull-requests-limit: 2
    ignore:
        -dependency-name: "minimatch"
        versions: ["^3.x"]

- package-ecosystem: "bundler"
  directory: "/"
  schedule:
    interval: "monthly"
    time: "07:57"
    timezone: "America/New_York"
open-pull-requests-limit: 2
ignore:
    -dependency-name: "nokogiri"
    versions: "~1.13.5"

In this example an 11ty site is used and we have specfified our npm package manager once again. We have set dependabot to check for version updates daily by specifying "daily" as the interval. We have set the branch that dependabot will run checks against as our staging branch by utilizing the "target-branch" option. By default, Dependabot will check for manifest files on the production branch and raise pull requests for version updates against it. You can override this behavior by explicitly setting the target-branch condition to another branch and have pull requests opened against it instead of your main branch which is what we have done here. It may be easier for Pages users to run more frequent version checks against a staging type branch as opposed to the main branch to make pull requests and alerts more manageable. We have also set Dependabot to ignore all version updates for 3.0 on the dependency rimraf with a pull request limit of 2.

version: 2
updates:
    -package-ecosystem: "npm"
    directory: "/"
    schedule:
        interval: "daily"
        timezone: "America/New_York"
        time: "10:30"
    target-branch: "staging"
    ignore:
        - dependency-name: "rimraf"
          versions: ["^3.x"]
    open-pull-requests-limit: 2

Please note that the dependencies used in this article were for example use only. We do not advocate for the explicit neglect of specific dependencies. It is up to each developers best judgement on how to best address version updates.

Sidecar processes are additional processes you can run in the same container as your application.

How do sidecar processes work?​

When deploying a sidecar with your application Cloud Foundry packages the code and configuration needed for running the application and sidecar in the same droplet. The droplet is then deployed in the same container with the application and sidecar being deployed in parallel. The application and sidecar have independent health checks.

Use Case​

Sidecars can be used on processes which depend on each other or need to run in the same container.

What are some example uses?​

Sidecar processes are useful for things such as application monitoring processes or a process that allows for a centralized server to deliver external configuration properties to an application and management of that configuration across different environments.

How do you deploy sidecar processes?​

You can push sidecar processes with your app by using one of two methods:

• Using an app manifest. For instructions, see Push an App with a Sidecar Using an App Manifest.
• With a custom buildpack. For instructions, see Sidecar Buildpacks.

What are some limitations and things to keep notice of?​

• The start and stop order of app processes and their sidecars is undefined.
• App processes and sidecars are codependent: If either crashes or exits, the other does also.
• Sidecars are currently not independently scalable. Sidecars share resources with the main application process and other sidecars within the container.
• Sidecars only support process-based health checks. HTTP health checks for sidecars are not currently supported.

Specific Java application requirements​

Linked here are specific considerations for Java applications.

On 3/28/2023, at approximately 10:35 AM ET, it was noted that due to an upstream CloudFoundry issue the CloudFoundry key is not accessible by typical wget or curl requests on Ubuntu using the CloudFoundry Command Line Interface. Despite the url being accessible by internet browser, when users attempt to use the wget command, it would be empty. Additionally curl requests would receive a 403 error / access denied error message. The work around for this issue is by providing a user agent with curl's -A option as noted in this issue ticket on the CloudFoundry github.

Status update and issue resolution​

On 3/28/2023, at approximately 6:18 PM ET, the upstream issue was addressed and resolved as stated in this issue ticket on the CloudFoundry github.

Prerequisites​

This process, outlined in the AWS documentation, will work with your Cloud.gov PostgreSQL database and S3 buckets, given the following:

• You have created a PostgreSQL RDS database with the aws-rds broker in your space.
• You have created an S3 bucket with the s3-broker service in your space.
• You have created a service-key and retrieve the credentials for your S3 bucket.

AWS S3 Extension​

First you must install the AWS S3 extension.

You will need to access the PostgreSQL database on the cli using one of the methods detailed here.

Information needed​

Once you have installed the AWS S3 extension, you then need to gather information to provide for the PostgreSQL RDS to S3 connection function which consist of:

• The name of the table on your RDS for PostgreSQL DB instance

  This is the table into which the aws_s3.table_import_from_s3 function is to import the data.

• The S3 bucket name

• The S3 file path

• The S3 file type

  These can be obtained from the service key for your S3 service. To get the credentials such as the bucket name, region and file type, these can be obtained from the service key, you can run cf service-key <SERVICE_INSTANCE_NAME> <KEY_NAME>; you will see a JSON description of the credentials. Treat these values as sensitive!

• The AWS Region: us-gov-west-1

• S3 Bucket Access Key Id and Secret obtained in your S3 service-key credentials

Permissions​

You will then set up permissions on your RDS for PostgreSQL DB instance to allow access to the file on the Amazon S3 bucket using the previously obtained credentials.

Importing data​

Finally you should be able to import the data into the table you have selected by issuing the aws_s3.table_import_from_s3 function as described here.

The status of encryption in transit between a customer application and a service instance is dependent on the service. Traffic from applications to ElastiCache and ElasticSearch has TLS enabled by default. The same is true for traffic to S3 as long as you are not using a public bucket in web server mode. For traffic to RDS databases, TLS is enabled but not enforced by default. Customers can require TLS in the code library that they are using to make connections to the RDS database.

Application to application​

While traffic between customer applications is not secured with TLS by default, customers can enable secure container networking with TLS between their applications on internal routes.

Inbound traffic to applications​

All inbound connections to customer applications on Cloud.gov are protected by TLS. As that traffic comes into Cloud.gov, it crosses a few boundaries where in memory it's decrypted for inspection/routing and then encrypted when it leaves that boundary all the way to the customer application endpoint. For further details, see the documentation on our SSL/TLS implementation.

The cause of the issue is that AWS CLI is overriding certificates that Cloud.gov provides and preventing the AWS CLI from being able to validate TLS connections, thus causing CLI commands to fail.

To force the AWS CLI to use the system CA cert stores, you can add this environment variable:

AWS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

Environment variables can be set for your application by using a manifest, the cf set-env CLI command, or in the .profile file for your application. For applications running custom Docker images, you could also set this environment variable in the Dockerfile for your image.

cf set-space-role <USERNAME> <ORG> <SPACE> <ROLE>

Where:

• USERNAME is the username of the user you want to add
• ORG is the name of your sandbox org
• SPACE is the name of the space you want the user to have access to within your org.
• ROLE is set to one of 'SpaceManager', 'SpaceDeveloper', or 'SpaceAuditor' depending on the level of access you want the user to have.

The next time the user logs back in to their account they will be able to see and access the space you assigned them to.

Unlike dedicated instance plans, which each provision a separate RDS database instance and take a while to create, shared instance plans re-used the same RDS database instance and created new databases within the same instance, which completed almost instantly.

You can replicate the behavior of the deprecated shared instance plans using an open-source service broker known as postgres-tinsmith.

Please note that while this approach has been tested manually, the postgres-tinsmith repo is not actively maintained by the Cloud.gov team, so this documentation is provided as a proof-of-concept only and does not offer any guarantees of production-readiness.

1. Create a dedicated RDS service​

View our available dedicated RDS instance plans for PostgreSQL:

cf marketplace -e aws-rds

Create a PostgreSQL service using one of the dedicated instance plans. For example:

cf create-service aws-rds micro-psql <dedicated-db-name>

2. Deploy and configure postgres-tinsmith​

Follow the instructions in the repo README to create and configure the postgres-tinsmith service broker.

Make sure to use the <dedicated-db-name> created in step 1 as the database service name to bind to postgres-tinsmith.

3. Create a database​

Once postgres-tinsmith is deployed and configured, create a database using the new shared plan of the postgres service:

cf create-service postgres shared <db-name>

4. Bind database to your application​

cf bind-service <application-name> <db-name>
cf restage <application-name>

Alternately, you can bind the database service to your application using a manifest.yml file.

Verifying your bound database service​

Running cf env <application> (e.g. cf env postgres-example-app) should show information about the bound <db-name> in the VCAP_SERVICES environment variable. See the CloudFoundry documentation for more information about VCAP_SERVICES.

5. Use the database in your application​

Make sure that your application properly reads and implements the credentials for the database from the VCAP_SERVICES environment variable.