chore: backport maintenance/doc changes and prepare release for v1.9.1 (#731)

* Add CBMC CI configuration (#709)

This commit adds a configuration file for the "CBMC Proofs" CI check.
This is in preparation for adding some custom check-out steps later.

* Use private submodules before CI run (#711)

* chore: Use continuous-integration environment for private submodule access (#714)

Co-authored-by: Robin Salkeld <[email protected]>

* chore: Add support policy (#720)

* Upgrade CBMC proof tools: starter kit and Litani 1.10.0 (#722)

* Upgrade proof tool submodules

This commit advances Litani to release 1.10.0, and the starter kit to
the tip-of-tree. This brings the following improvements:

- Profiling
    - Litani measures the memory usage of the CBMC safety checking and
      coverage checking jobs
    - The dashboard includes box-and-whisker diagrams for memory use per
      proof
    - The dashboard includes a graph of how many parallel jobs are
      running over the whole run, making it easy to choose a CI machine
      with enough parallelism
    - It is now possible to designate particular proofs as "EXPENSIVE";
      Litani runs expensive proofs serially, ensuring that they do not
      over-consume resources like RAM.

- UI improvements
    - Each pipeline page includes a table of contents
    - Each pipeline page includes a dependency graph of the pipeline
    - Each job on the pipeline page has a hyperlink to that job
    - The terminal output is now less noisy

* Change cbmc-batch.yaml to cbmc-proof.txt

This makes the proof layout consistent with the starter kit, which will
allow us to use a generic run script in a future commit. Putting this
in commit by itself because the diff is huge and not worth reading (just
moving some files and changing two lines in the runscript).

* Symlink run-cbmc-proofs.py to starter kit

The run script is now a symbolic link into the starter kit submodule,
meaning that it will be updated whenever the starter kit is. This is
done iso that E-SDK doesn't carry custom modifications to the run script
unless necessary; previous commits have made the E-SDK proofs consistent
with the generic starter kit conventions.

* fix: Simplify / update build instructions. (#713)

Co-authored-by: June Blender <[email protected]>
Co-authored-by: Alex Chew <[email protected]>

* fix(proof_timeout): mark high-memory proofs expensive (#710)

* Removed OOM test, as OOM is no longer possible from aws allocators (#728)

* chore: pin newer aws-sdk-cpp in macOS CI builds (#729)

* chore: update version number and changelog for v1.9.1

* chore: update CBMC CI submodules

Co-authored-by: Kareem Khazem <[email protected]>
Co-authored-by: Robin Salkeld <[email protected]>
Co-authored-by: Ben Farley <[email protected]>
Co-authored-by: lizroth <[email protected]>
Co-authored-by: June Blender <[email protected]>
Co-authored-by: Justin Boswell <[email protected]>

Author: 7 people

.github/workflows/osx.yml

@@ -7,6 +7,7 @@ on: ["pull_request", "push"]
 jobs:
   OSX:
     runs-on: ${{ matrix.os }}
+    environment: continuous-integration
 
     strategy:
       matrix:
@@ -20,13 +21,22 @@ jobs:
         uses: actions/checkout@v2
         with:
           submodules: recursive
+        if: ${{ github.repository == 'aws/aws-encryption-sdk-c' }}
+
+      - name: Checkout PR with CI bot token
+        uses: actions/checkout@v2
+        with:
+          submodules: recursive
+          token: ${{ secrets.CI_BOT_TOKEN }}
+        if: ${{ github.repository == 'aws/private-aws-encryption-sdk-c-staging' }}
 
       - name: Checkout AWS C++ SDK
         uses: actions/checkout@v2
         with:
           repository: "aws/aws-sdk-cpp"
-          ref: "1.8.32"
+          ref: "1.9.124"
           path: "aws-sdk-cpp"
+          submodules: recursive
 
       - name: Install dependencies
         run:

CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## 1.9.1 -- 2021-10-20
+
+### Maintenance
+
+* Add [support policy](https://github.com/aws/aws-encryption-sdk-c/blob/master/SUPPORT_POLICY.rst)
+* CBMC CI upgrades ([#709](https://github.com/aws/aws-encryption-sdk-c/pull/709), [#710](https://github.com/aws/aws-encryption-sdk-c/pull/710), [#711](https://github.com/aws/aws-encryption-sdk-c/pull/711), [#722](https://github.com/aws/aws-encryption-sdk-c/pull/722), [#726](https://github.com/aws/aws-encryption-sdk-c/pull/726))
+* Simplify / update build instructions ([#713](https://github.com/aws/aws-encryption-sdk-c/pull/713))
+* Remove OOM test, as OOM is no longer possible from aws allocators ([#728](https://github.com/aws/aws-encryption-sdk-c/pull/728))
+* Pin newer aws-sdk-cpp in macOS CI builds ([#729](https://github.com/aws/aws-encryption-sdk-c/pull/729))
+
 ## 1.9.0 -- 2021-05-27
 
 * Improvements to the message decryption process.

CMakeLists.txt

@@ -51,7 +51,7 @@ set(PROJECT_NAME aws-encryption-sdk)
 # Version number of the SDK to be consumed by C code and Doxygen
 set(MAJOR 1)
 set(MINOR 9)
-set(PATCH 0)
+set(PATCH 1)
 
 # Compiler feature tests and feature flags
 set(USE_ASM TRUE

README.md

@@ -0,0 +1,33 @@
+Overview
+========
+This page describes the support policy for the AWS Encryption SDK. We regularly provide the AWS Encryption SDK with updates that may contain support for new or updated APIs, new features, enhancements, bug fixes, security patches, or documentation updates. Updates may also address changes with dependencies, language runtimes, and operating systems.
+
+We recommend users to stay up-to-date with Encryption SDK releases to keep up with the latest features, security updates, and underlying dependencies. Continued use of an unsupported SDK version is not recommended and is done at the user’s discretion
+
+
+Major Version Lifecycle
+========================
+The AWS Encryption SDK follows the same major version lifecycle as the AWS SDK. For details on this lifecycle, see  `AWS SDKs and Tools Maintenance Policy`_.
+
+Version Support Matrix
+======================
+This table describes the current support status of each major version of the AWS Encryption SDK for C. It also shows the next status each major version will transition to, and the date at which that transition will happen.
+
+.. list-table::
+    :widths: 30 50 50 50
+    :header-rows: 1
+
+    * - Major version
+      - Current status
+      - Next status
+      - Next status date
+    * - 1.x
+      - Maintenance
+      - End of Support
+      - 2022-07-08
+    * - 2.x
+      - Generally Available
+      -
+      -
+
+.. _AWS SDKs and Tools Maintenance Policy: https://docs.aws.amazon.com/sdkref/latest/guide/maint-policy.html#version-life-cycle

SUPPORT_POLICY.rst

@@ -24,22 +24,6 @@ using namespace Aws::Cryptosdk::Testing;
 
 const char *TEST_STRING = "Hello World!";
 
-static void *s_bad_malloc(struct aws_allocator *allocator, size_t size) {
-    return NULL;
-}
-
-static void s_bad_free(struct aws_allocator *allocator, void *ptr) {}
-
-static void *s_bad_realloc(struct aws_allocator *allocator, void *ptr, size_t oldsize, size_t newsize) {
-    return NULL;
-}
-
-static struct aws_allocator default_bad_allocator = { s_bad_malloc, s_bad_free, s_bad_realloc };
-
-struct aws_allocator *t_aws_bad_allocator() {
-    return &default_bad_allocator;
-}
-
 int awsStringFromCAwsByteBuf_validInputs_returnAwsString() {
     struct aws_byte_buf b = aws_byte_buf_from_c_str(TEST_STRING);
     Aws::String b_string  = aws_string_from_c_aws_byte_buf(&b);
@@ -143,16 +127,6 @@ int appendKeyToEdks_appendSingleElement_elementIsAppended() {
     return 0;
 }
 
-int appendKeyToEdks_allocatorThatDoesNotAllocateMemory_returnsOomError() {
-    struct aws_allocator *oom_allocator = t_aws_bad_allocator();
-    EdksTestData ed;
-    TEST_ASSERT_ERROR(
-        AWS_ERROR_OOM,
-        t_append_c_str_key_to_edks(
-            oom_allocator, &ed.edks.encrypted_data_keys, &ed.enc, ed.data_key_id, ed.key_provider));
-    return 0;
-}
-
 int appendKeyToEdks_multipleElementsAppended_elementsAreAppended() {
     EdksTestData ed1;
     EdksTestData ed2("enc2", "dk2", "kp2");
@@ -234,7 +208,6 @@ int main() {
     RUN_TEST(awsStringFromCAwsByteBuf_validInputs_returnAwsString());
     RUN_TEST(awsUtilsByteBufferFromCAwsByteBuf_validInputs_returnAwsUtils());
     RUN_TEST(appendKeyToEdks_appendSingleElement_elementIsAppended());
-    RUN_TEST(appendKeyToEdks_allocatorThatDoesNotAllocateMemory_returnsOomError());
     RUN_TEST(appendKeyToEdks_multipleElementsAppended_elementsAreAppended());
     RUN_TEST(awsStringFromCAwsString_validInputs_returnAwsString());
     RUN_TEST(awsMapFromCAwsHashHable_hashMap_returnAwsMap());

aws-encryption-sdk-cpp/tests/unit/t_cpputils.cpp

@@ -0,0 +1,28 @@
+# Configuration for the "CBMC Proofs" CI
+#
+# What the CI should do, depending on what branch the pull request
+# targets. This is a list of branch names (or '*' as a wildcard that
+# matches all branches), with an associated action. Actions can be:
+#
+#   name: run-proofs
+#
+#   or
+#
+#   name: skip
+#   message: "A message to post to GitHub about why the branch was skipped"
+#   status: <"success"|"failure">
+#           (whether the GitHub status check should succeed or fail)
+#
+behaviors:
+
+  - target-branches:
+    - '*'
+    action:
+      name: run-proofs
+
+
+checkout-script:
+  # If this is the private version of the repository, we need to pull in
+  # the private versions of the submodules.
+  - "echo Originating GitHub repository: ${GITHUB_REPOSITORY}"
+  - ./switch-private-submodules --verbose env

cbmc-ci/ci-config.yaml

@@ -0,0 +1,103 @@
+#!/usr/bin/env python3
+#
+# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"). You
+# may not use this file except in compliance with the License. A copy of
+# the License is located at
+#
+# http://aws.amazon.com/apache2.0/
+#
+# or in the "license" file accompanying this file. This file is
+# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+# ANY KIND, either express or implied. See the License for the specific
+# language governing permissions and limitations under the License.
+
+
+import argparse
+import logging
+import os
+import subprocess
+import sys
+
+
+DESCRIPTION = "Switch between public and private versions of submodules"
+
+MODULES = [{
+    "submodule": "aws-encryption-sdk-cpp/tests/test_vectors/aws-encryption-sdk-test-vectors",
+    "private": "https://github.com/awslabs/private-aws-encryption-sdk-test-vectors-staging.git",
+    "public": "https://github.com/awslabs/aws-encryption-sdk-test-vectors.git",
+}, {
+    "submodule": "aws-encryption-sdk-specification",
+    "private": "https://github.com/awslabs/private-aws-encryption-sdk-specification-staging.git",
+    "public": "https://github.com/awslabs/aws-encryption-sdk-specification.git",
+}]
+
+
+def switch_to(version):
+    logging.info("Switching to %s version of the submodules", version)
+    for module in MODULES:
+        cmd = [
+            "git", "config",
+            f'url."{module[version]}".insteadOf',
+            module["public"],
+        ]
+        logging.info(" ".join(cmd))
+        subprocess.run(cmd, check=True)
+
+    subprocess.run(["git", "submodule", "sync"], check=True)
+    subprocess.run([
+        "git", "submodule", "update", "--init", "--recursive", "--checkout"],
+        check=True)
+
+
+def switch_to_env(_):
+    repo = os.getenv("GITHUB_REPOSITORY")
+    if not repo:
+        logging.error(
+            "Could not determine which submodules to check out "
+            "($GITHUB_REPOSITORY is not set).")
+        sys.exit(1)
+
+    if repo == "aws/private-aws-encryption-sdk-c-staging":
+        switch_to("private")
+    else:
+        switch_to("public")
+
+
+OPERATIONS = {
+    "public": switch_to,
+    "private": switch_to,
+    "env": switch_to_env,
+}
+
+
+def main():
+    pars = argparse.ArgumentParser(description=DESCRIPTION)
+    for arg in [{
+            "flags": ["operation"],
+            "choices": list(OPERATIONS.keys()),
+            "default": "public",
+            "help": "Switch to public or private versions of the submodules, "
+                    "or decide which by reading the $GITHUB_REPOSITORY "
+                    "environment variable. Default: %(default)s."
+        }, {
+            "flags": ["-v", "--verbose"],
+            "action": "store_true",
+            "help": "verbose output",
+    }]:
+        flags = arg.pop("flags")
+        pars.add_argument(*flags, **arg)
+    args = pars.parse_args()
+
+    fmt = "switch-private-submodules: %(message)s"
+    if args.verbose:
+        logging.basicConfig(format=fmt, level=logging.INFO)
+    else:
+        logging.basicConfig(format=fmt, level=logging.WARNING)
+
+    OPERATIONS[args.operation](args.operation)
+
+
+if __name__ == "__main__":
+    main()

switch-private-submodules

@@ -3,6 +3,7 @@ proofs/**/logs
 proofs/**/gotos
 proofs/**/report
 proofs/**/html
+proofs/output
 
 # Emitted by CBMC Viewer
 TAGS-*

verification/cbmc/.gitignore

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof

verification/cbmc/jobs_dontrun/aws_cryptosdk_hdr_clean_up/cbmc-batch.yaml

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof

verification/cbmc/jobs_dontrun/aws_cryptosdk_hdr_clean_up/cbmc-proof.txt

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof

verification/cbmc/jobs_dontrun/aws_cryptosdk_hdr_size/cbmc-batch.yaml

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof

verification/cbmc/jobs_dontrun/aws_cryptosdk_hdr_size/cbmc-proof.txt

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof

verification/cbmc/jobs_dontrun/aws_cryptosdk_hdr_write/cbmc-batch.yaml

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof

verification/cbmc/jobs_dontrun/aws_cryptosdk_hdr_write/cbmc-proof.txt

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof

verification/cbmc/jobs_dontrun/hdr_zeroize/cbmc-batch.yaml

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof

verification/cbmc/jobs_dontrun/hdr_zeroize/cbmc-proof.txt

@@ -0,0 +1 @@
+This file marks the directory as containing a CBMC proof