Give me a name for an article about microservices on AWS.
Example output:
"Scaling New Heights: Mastering Microservices Architecture on AWS"
Give me a name for an article about microservices on AWS.
Example name: Using microservices on AWS to create better domain boundaries.
Example output:
"Architecting Success: Harnessing Microservices on AWS for Domain-Driven Design"
Give me a name for an article about microservices on AWS.
Examples:
- Using grapes and microservices on AWS to create better domain boundaries
- Microservices, DDD, and pineapple help you clarify your business domains
- What fruit syrup and microservices have to do with your business
"Fruitful Architecture: Blending Microservices with AWS and a Dash of DDD for Sweeter Domain Boundaries"
Summarize the text below.
"""
Secrets Management in the Cloud
Solving the run-time vs compile-time conundrum.
By Mikael Vesavuori
Published in ITNEXT on Nov 11, 2023
A somewhat popular brain teaser of a software architecture question is:
“Where do I store my secrets?”
First things first: What is a secret? A secret is confidential and sensitive information that needs to be protected from unauthorized access or disclosure. Typical examples of this could include passwords, encryption keys, API tokens, and other data that, if exposed, could compromise security or privacy. You will have an awful time creating secure software if you aren’t using secrets. Moreover, we want to disambiguate a secret from a mere configuration value as they are often conflated—more on that in a bit.
Many software engineers will have read The Twelve-Factor App and its Config section. Unfortunately, for many of us working today and with tooling that didn’t exist back then, some of those recommendations have gone sour with time. When it comes to secrets management, this part of the 12-factor concept doesn’t quite hold up in all cases.
Let’s therefore pick the question apart and look at some better options.
Some notes before we start:
While I don’t use a lot of Docker, there is a bit of reading to do concerning secrets management in that, too, which I won’t cover here.
Over on LinkedIn, the option of using sealed-secrets was mentioned if you are in the Kubernetes space.
How to store the secret in the first place?
Let’s first figure out how to store the secret.
You really don’t want to check in secrets
There are so many stories around lax development practices leading to both small and huge data leaks because of developers making their secrets available to anyone with access to the source code.
A convention that exists among developers is using files, such as .env, to store configuration values. Using .gitignore they’ll ensure Git won’t pick up that data. But where do they get the values from initially?
One option is that they store it in an insecure environment, like in their local notetaking app, in a shared documentation surface, or in direct messaging tools like Slack. For insensitive parameters this is fine.
Another option is that they store them in a secure environment, such as a collectively accessible password manager, such as Bitwarden or 1Password. Using group-based and personal restrictions, you end up with a great way to store access tokens and other sensitive data.
Generally, we can establish that checking in stuff like appsettings.json shouldn’t be an issue, given the values contained therein are of a generic, insensitive nature; literally being application settings. Keep doing this, as per the Twelve Factor advice!
Infrastructure-as-code as a (partial) solution
When managing cloud resources, use Infrastructure as Code (IAC) tools like Terraform, AWS CloudFormation, or Azure Resource Manager templates. The idea is to store secrets separately from your IAC code and use the respective secrets management tools to inject them into the application at runtime.
There’s a small catch-22 though. You’ll have to somehow originally assign the secret’s value to the infrastructural resource. I haven’t personally figured out a water-tight way to add the actual, real secret into cloud environments without doing this manually, in order to avoid the secret being stored in source code somewhere. I’m open for the possibility that such options exist, but I don’t see anything dramatically bad using, for example, a password manager such as Bitwarden, Proton Pass, or 1Password to contain these “origin” secrets. Thus: Create infrastructure for secrets with IAC but add the values to the resources manually. It’s not perfect, but it’s a clean way.
Breaking down the original question
Back to the starting question.
To solve it, we can use a few proxy questions to better understand the nature of our use case.
The first thing we have to do is understand the actual level of secrecy (I know, I tried other words!):
Is the secret actually more of a configuration value or parameter? In this case it’s hardly a “secret”, is it? 😉
Is the secret “kind-of-secret”, such as a commonly known value (shared tokens come to mind) in your organization? These shouldn’t leak, but it’s not the end of the world if they do, because you hopefully have a plan to rotate them quickly and painlessly.
Is the secret owned by an individual user? In which case it’s highly secret and leakage would break the confidentiality (and trust) of your system.
The second aspect is understanding the level of uniqueness in the secret:
Is the secret completely static and unchanging (at least over an extended period of time)?
Is it the same secret being used deterministically across all cases, such as a secret that depends on the stage (i.e. DEV, QA, PROD)? In this case there is a bit of logical/structural coupling on the values.
Is the secret pulled and used uniquely for a given actor, say a user? In this case we can’t know, or possibly even access, the secret before-hand.
The third question concerns context isolation:
Is the value shared across multiple contexts (i.e. multiple applications)?
Does the secret have to be validated at the time of an operation being performed or is it enough to have a precompiled response?
How does secrets rotation impact systems? Is it contained to a single system or will multiple systems need to update?
Secrets don’t come in one size
Our question can now be rephrased into asking when the secret should be accessed:
Do we need it at compile-time?
Do we need it at run-time?
Compile-time
Compile-time secrets are configuration values that are known and set during the software development and build process. These secrets are typically stored securely (encrypted) and used to compile the code or build application artifacts.
An example of such usage is baking in secrets in GitHub, GitLab, Azure DevOps, or any other such tool. I’ll give two examples to contextualize.
An abbreviated example using a GitHub workflow could look like this:
name: main
[...]
jobs:
deploy:
uses: my_org/my_repo/.github/workflows/deploy.yml@main
secrets:
aws-role-arn: ${{ secrets.AWS_ROLE_ARN }}
This uses GitHub’s built-in secrets management where the value can only be read back by the CI workflow.
Another example, using Serverless Framework (version 3) to fetch a secret in AWS Secrets Manager:
service: my-service
[...]
custom:
config:
# In SLS, this needs to be set as a reference before we can get the actual value
SOME_CREDENTIALS: ${ssm:/aws/reference/secretsmanager/MyThingHere}
# Now that the reference is set, we can point to the field in the secret
someValue: ${self:custom.config.SOME_CREDENTIALS.SOME_VALUE}
During the compilation phase, Secrets Manager will be called and the value will be baked in according to your configuration. Ergo, if the person/machine calling the compilation is not permitted to access Secrets Manager and the specific secret, then it’s going to fail.
These secrets are ultimately made available via environment variables. This is nice and all, but it’s not necessarily very secret. Compile-time secrets are (in a way) really only secret for you or other people with access to the build environment—in the actual binary, it’s all going to be there in the clear. It’s quite common that you won’t be able to optically read back the secret values without resetting them.
In terms of pros and cons, it goes something like so:
✅ The simplest option
✅ The fastest option
✅ Free or essentially free
❌ Should only be used for statically known, mostly insensitive values
❌ Secret is coupled to the time of compilation; rotation requires rebuild and means potential downtime
❌ Assume anything baked-in during compile time can be exposed; this issue can be somewhat mitigated from attacks from the general public if the system in question is a back-end or otherwise “private” system
Side note: Using the CI tool or an external tool for compile-time secrets
Then what about storing compile-time secrets in GitHub and/or an external system, like Hashicorp Vault or AWS Secrets Manager? Should you use one, both, neither?
The way I would recommend it is:
Use your CI environment to host configuration values and mostly insensitive values as “secrets”. This minimizes your pain in accessing them. The CI environment tends to be shared and thus settings and so on can be seen by other teams, so assume people will have a possibility to see, if not the values, then how the code actually looks and retrieves values. You can typically have both shared/organizational secrets as well as repository-specific secrets, making it a flexible and simple option for the basic use cases.
Using a separate tool like AWS Secrets Manager, especially when combined with things like workload-separated AWS accounts, can help you isolate secrets into a more gated environment where your team has greater access as well as the ability to actually read back the values. That’d result in you not needing yet another layer of password management—because, after all, where do you store the values that go into CI as you can’t retrieve them once they are in the system? With this approach you can opt for a shared keystore or decentralize it with teams/systems owning their own keystores.
In practical terms, I’d put all the basic baseline configuration into GitHub and I’d move anything I want to reuse—say, an important token that is shared between multiple repos within the same domain—into a keystore like Secrets Manager.
Run-time
Run-time secrets are dynamic and sensitive values that are required to be retrieved and used during the execution of the application. They are typically not known during compile time and need to be securely managed at runtime.
Common options include Hashicorp Vault, AWS Secrets Manager, Google Cloud Secret Manager and Azure Key Vault.
Using the AWS option, the below snippet would request the secret called “MySecret” during run-time.
import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager";
// Setup
const config = {
region: "eu-north-1",
secretName: "MySecret"
}
const client = new SecretsManagerClient({ region: config.region });
// Send command
const input = { SecretId: config.secretName };
const command = new GetSecretValueCommand(input);
const response = await client.send(command);
// Get value
const secretString = response.SecretString;
const secret = JSON.parse(secretString);
console.log('Retrieved secret:', secret);
With this approach, it’s imperative that the requesting party (say, the IAM role) has permissions to access the secret—but this rigor is also part of the magic of such solutions.
The logic around access to secrets is effectively pushed to the application. If you don’t need that in the application, this option won’t be right for you.
Pros and cons for run-time secrets:
✅ The most secure option and can be tuned to a much finer level of access
✅ The only reasonable option when you have more complex secrets setups (such as per user)
✅ Decouples the secret from the runtime/application
✅ Ensures correct (in time) validation and use of the secret
❌ Adds a bit of latency
❌ Adds cost
❌ Adds infrastructure/tooling
Diagram
I’ve provided the above questions as a, hopefully, helpful diagram. You can substitute the GitHub and AWS Secrets Manager icons to any equivalent product.
Note that the green boxes (using both icons) represent that either solution could be applicable.
Given that these three questions are not individual tracks but depend on each other, you’ll have to weigh approaches once you at least clarified the rough parameters of your problem space.
Secrets management doesn’t have to be that hard and it gets a lot easier when you have a mental model to base your architecture on. Using this mental model should assist you in many of the common cases you’ll face in cloud-based software development.
Go and build some secure applications now, will ya? 🍻
"""
Example output:
The article, "Secrets Management in the Cloud," authored by Mikael Vesavuori and published in ITNEXT, delves into the challenges and strategies of managing sensitive data, or 'secrets', in software development, particularly in cloud environments. Vesavuori begins by defining secrets as confidential information like passwords and API tokens, emphasizing the distinction from mere configuration values. He critiques the outdated recommendations from The Twelve-Factor App in the context of modern tooling and explores more contemporary solutions.
The article discourages the practice of storing secrets in source code due to the high risk of data leaks, suggesting instead the use of secure environments like password managers. Vesavuori then introduces Infrastructure as Code (IAC) as a partial solution, proposing the separation of secrets from IAC code and manual addition of secret values to infrastructure to prevent them from being stored in source code.
The author breaks down the problem further by considering the secrecy level, uniqueness, and context isolation of secrets, leading to the core question of whether secrets are needed at compile-time or run-time. For compile-time secrets, he discusses their usage, security, and limitations, including their vulnerability when embedded in the build environment. He recommends using CI environments for hosting configuration values and sensitive values, with an option for tools like AWS Secrets Manager for greater security.
For run-time secrets, Vesavuori describes them as dynamic and sensitive, requiring secure management during application execution. He details the usage of services like AWS Secrets Manager and emphasizes the importance of proper permissions for accessing these secrets.
The article concludes with a diagram to aid in deciding between compile-time and run-time secrets management, based on the level of secrecy, uniqueness, and context isolation. Vesavuori encourages the use of a structured mental model for secrets management to simplify cloud-based software development and ensure the creation of secure applications.
Summarize the text again but:
- Do it in two paragraphs
- The word count must be around 175-225 words
- Use simple language that a non-technical reader can understand
Example output:
The article "Secrets Management in the Cloud" by Mikael Vesavuori talks about how to keep sensitive information safe in software development, especially when using cloud services. This information, known as 'secrets', includes things like passwords and special codes that allow access to certain parts of a system. Vesavuori explains that keeping these secrets secure is crucial because if they get out, they can cause serious security problems. He points out that some traditional methods for managing these secrets are outdated and don't work well with newer tools and technology. Instead of storing secrets directly in the code where they can be easily found, he suggests using secure storage places like password managers. He also mentions a method called Infrastructure as Code (IAC), which helps manage and set up cloud resources securely, but notes that it's tricky to do this without exposing secrets at some point.
Vesavuori dives deeper into the topic by looking at different types of secrets and how their importance and usage can vary. He distinguishes between secrets needed during the initial setup of a program (compile-time) and those needed while the program is actually running (run-time). For compile-time secrets, he discusses how they can be safely incorporated during the development process, although this method has its limitations and risks. For run-time secrets, he emphasizes the importance of dynamic and secure access, recommending specific tools and practices to manage these secrets effectively. The article ends with a guide to help decide the best approach for managing secrets, encouraging a structured and thoughtful way of keeping sensitive information safe in the digital space.
You are an expert software engineer with decades of training experience.
Summarize the text again in fewer than 200 words and in the style of Ricky Gervais.
Example output:
In the article "Secrets Management in the Cloud," Mikael Vesavuori chats about the comedy of errors that is trying to keep your digital secrets actually secret in the world of cloud computing. He kicks off by asking, "Where do I put my secrets?" and not in the 'hiding the Christmas presents from the kids' kind of way. We're talking passwords, API keys—the juicy stuff. He's quick to chuckle at the outdated advice from The Twelve-Factor App, saying it's about as fresh as last week's sushi.
Mikael gets real about the mess developers make by leaving their secrets in the code like dirty laundry on the floor. His fix? Use a password manager or some fancy Infrastructure as Code tools, but even then, it's like trying to sneak a snack in a quiet cinema - risky. Then, he breaks it down: are your secrets really secret or just pretending? Compile-time, run-time - it's all a game of when and where to spill the beans safely. He wraps up with a diagram that's more like a 'choose your own adventure' for secrets, nudging us to be a bit smarter with our digital whispers. All in all, Mikael's telling us, keeping secrets isn't just about locking them away; it's about being clever and a bit cheeky with how we handle them.
You are an expert software engineer doing code reviews.
You will be supplied with code. For each snippet, you will do a code review of it, using conventional best practices such as (but not limited to) SOLID. Return the review back as a JSON object in the following shape:
"""
{
"quality_score": <numeral rating of code quality between 0 and 100>,
"defects": [
<array of major issues encountered>
],
"comments": <short text description of why the code was scored this way>
}
"""
Do not output explanations or anything other than the JSON object.
"""
export function bytes(str: string) {
return new Blob([str]).size;
}
"""
Example output:
{
"quality_score": 40,
"defects": [
"Lack of error handling",
"Lack of type checking or validation"
],
"comments": "The code calculates the size of a Blob containing the input string. However, it lacks proper error handling and type checking, which can lead to unexpected behavior or errors in some cases. It's important to handle edge cases and validate inputs for better code quality."
}You are an expert software architect supporting with refactoring a legacy codebase to a new language. It's imperative that the old code is also improved and tested.
The old language is Python and the new one is Typescript. The test tool is Jest.
You will be supplied with code. For each snippet you will do the following steps:
1. Make a code review of the current solution. Verify the code review with a peer before settling on the review results.
2. Write an improved version of the code in the new language. It most be idiomatic and follow the language's conventions and best practices.
3. Create appropriate unit tests using common testing best practices.
You will output the review as a bulleted list and the code and test in their respective file formats.
import random
import string
def extract_names(input_object):
if type(input_object) != dict or "first_name" not in input_object or "last_name" not in input_object:
return None
first_name = input_object["first_name"]
last_name = input_object["last_name"]
new_object = {
"firstName": first_name,
"lastName": last_name,
"transactionId": ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(10)),
"status": "IN_PROGRESS"
}
return new_objectExample output:
Let's start by reviewing the provided Python code and then refactoring it into TypeScript while improving and testing it.
**Code Review:**
1. The function `extract_names` takes an `input_object` as an argument.
2. It checks if the input is a dictionary and if it contains both "first_name" and "last_name" keys.
3. If the input is not a valid dictionary or if the required keys are missing, it returns `None`.
4. It extracts the "first_name" and "last_name" values from the input.
5. It creates a new dictionary with the extracted names, a randomly generated "transactionId," and "status" set to "IN_PROGRESS."
6. Finally, it returns the new dictionary.
**Refactoring into TypeScript:**
Now, let's refactor this code into TypeScript, following the language's conventions and best practices.
```typescript
import * as randomstring from 'randomstring';
interface InputObject {
first_name: string;
last_name: string;
}
interface NewObject {
firstName: string;
lastName: string;
transactionId: string;
status: string;
}
function extractNames(inputObject: InputObject | Record<string, unknown>): NewObject | null {
if (
typeof inputObject !== 'object' ||
!('first_name' in inputObject) ||
!('last_name' in inputObject)
) {
return null;
}
const { first_name, last_name } = inputObject as InputObject;
const newObject: NewObject = {
firstName: first_name,
lastName: last_name,
transactionId: randomstring.generate(10),
status: 'IN_PROGRESS',
};
return newObject;
}
Changes Made:
- Imported the
randomstringlibrary to generate random strings. - Defined TypeScript interfaces for
InputObjectandNewObjectto ensure type safety. - Modified the function signature to specify input and output types.
- Improved type checking and casting for input validation.
- Used TypeScript naming conventions (camelCase for variables).
- Used
randomstring.generateto generate a random transactionId.
Unit Testing with Jest:
Now, let's create appropriate unit tests using Jest for the refactored TypeScript code. Create a separate file for the tests, e.g., extractNames.test.ts.
import { extractNames } from './yourFileName'; // Replace 'yourFileName' with the actual file name
import * as randomstring from 'randomstring';
describe('extractNames', () => {
it('should return null when input is not a dictionary', () => {
const result = extractNames(null);
expect(result).toBeNull();
});
it('should return null when input is missing "first_name" or "last_name"', () => {
const input1 = { first_name: 'John' };
const input2 = { last_name: 'Doe' };
const input3 = {};
expect(extractNames(input1)).toBeNull();
expect(extractNames(input2)).toBeNull();
expect(extractNames(input3)).toBeNull();
});
it('should return a new object with correct properties', () => {
const input = { first_name: 'John', last_name: 'Doe' };
const result = extractNames(input);
expect(result).toEqual({
firstName: 'John',
lastName: 'Doe',
transactionId: expect.any(String),
status: 'IN_PROGRESS',
});
expect(result.transactionId.length).toBe(10);
});
});Note:
- Replace
'yourFileName'with the actual file name where theextractNamesfunction is defined. - These tests cover cases where the input is not a dictionary, where the required keys are missing, and where the function returns the expected output with correct properties.
Make sure to run these tests using Jest to verify the correctness of your refactored TypeScript code.