Look into reserving instances on AWS

[hellais]

If we make use of reserved instances in AWS we can end up saving up to 50% of infra costs.

We should:

• Make an inventory of all instances we currently have and document which ones we expect to have for the forseable future
• Make a list of which ones we are comfortable with reserving for some amount of time
• Understand what happens if we decide we actually needed bigger ones and how the convertible vs reserved thing works

[LDiazN]

Inventory

• monitoring proxy: Used for proxying monitoring requests to hosts behind AWS, including static hosts (clickhouseproxy, fastpath) and ECS clusters. Current instance type: t3a.nano
• clickhouse proxy: Similar to the monitoring proxy. Provides access to clickhouse to hosts behind AWS. Current instance type: t3a.nano
• fastpath: The host running the fastpath process and the api uploader. Current instance type: c6i.large
• Cluster Machines (x8): Hosts running ECS tasks. Current instance type: t3a.medium
• Codesign box: Current instance type: t3.micro
• Ansible controller box: Currenly unused AFAIK, t2.micro

Machines that could be reserved

• monitoring proxy and clickhouse proxy: Good candidates to be reserved, they are static and we need them for basic services. We might increase the machine size before reserving tho, since it's cheaper and that way we don't have to be afraid of resizing them after reserving

• codesign and fastpath: We can just reserve them as they are. The fastpath shouldn't be moved that often to avoid breaking the measurement stream, and we aim to deploy it as an ECS service in the future

• Cluster machines: These are spawned automatically depending on traffic, but we could reserve a couple of them since we always need a minimum capacity anyways, while keeping the rest as dynamic

[LDiazN]

Resizing instances and convertible vs standard

According to this doc it seems like the only difference between convertibles and standard instances is the type of flexibility they offer:

Standard reserved instances

Customers have the flexibility to change the Availability Zone, the instance size, and networking type of their Standard Reserved Instances.

Convertible reserved instances

additional flexibility, such as the ability to use different instance families, operating systems, or tenancies over the Reserved Instance term

So it seems that the main difference between convertible and standard is the ability to change OS and family (family is described with the first letter of the machine name: t3.small => family == t)

There also seems to be some rules on resizing. Since you reserve a capacity (e.g. t3.medium), when resizing you can go up or down.

If you go up, we pay part of the bigger machine with the reserved contract: price(t3.large) == reserved_price(t3.medium) + something

If you go down, you can get several machines depending on the reserved one:
price(t3.small) == reserved_price(t3.medium) / 2 meaning that we can have 2 small ones from one medium

More details about this here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/apply_ri.html

Here's the table they use for this conversions:

Instance size	Normalization factor
nano	0.25
micro	0.5
small	1
medium	2
large	4
xlarge	8

(there are more rows but these ones are the most interesting to us)