Skip to content

rfc: A new tool to implement memory stress #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
FingerLeader wants to merge 3 commits into
base: main
Choose a base branch
from
Open

rfc: A new tool to implement memory stress #25

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e8d08ac
rfc of a new tool to implement memory stress
FingerLeader Oct 29, 2021
cde0b95
Merge branch 'main' of https://github.com/FingerLeader/rfcs
FingerLeader Oct 29, 2021
9cb0ae2
rfc of a new tool to implement memory stress
FingerLeader Oct 29, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions text/2021-10-11-self-made-memory-stress-program.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Self-made Memory Stress Program

## Summary

Create a program to replace stress-ng to realize mem stress.

## Motivation

Because stress-ng realize memory stress by applying read and write pressure instead of occupying memory space, which will affect CPU performance, a program purely takes up memory space is need to generate a memory StressChaos.

## Detailed design

Using `syscall.Mmap()` to allocate memory space directly, so that uncertain footprint can be avoided. And write 1 byte on every virtual memory page of the space we allocated can achieve our goal.

```data, err := syscall.Mmap(-1, 0, length, syscall.PROT_READ|syscall.PROT_WRITE, syscall.MAP_PRIVATE|syscall.MAP_ANONYMOUS)```
Copy link
Member

@cwen0 cwen0 Nov 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

data, err := syscall.Mmap(-1, 0, length, syscall.PROT_READ|syscall.PROT_WRITE, syscall.MAP_PRIVATE|syscall.MAP_ANONYMOUS)

Copy link
Member

@cwen0 cwen0 Nov 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add more introduction about syscall.PROT_READ|syscall.PROT_WRITE, syscall.MAP_PRIVATE|syscall.MAP_ANONYMOUS

Copy link
Member Author

@FingerLeader FingerLeader Nov 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter syscall.PROT_READ|syscall.PROT_WRITE let us read and write the array data, so we can write some bytes in it to occupy memory page.
And the parameter syscall.MAP_PRIVATE makes the mapping not visible to other processes and are not carried through to the underlying file, syscall.MAP_ANONYMOUS makes the mapping won't be backed by any file. Therefore, the allocated memory will not leave any mark or be accessed.
You can get more information at https://man7.org/linux/man-pages/man2/mmap.2.html.

Copy link
Member

@cwen0 cwen0 Nov 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to add this introduction on RFC


Furthermore, users can specify time to let the memory allocation increase linearly, so they can simulate a memory leak. The program will write certain numbers of pages in a time interval longer than 100ms , which can reduce the impact of the delay of `time.Sleep()`.

## Alternatives

- stress-ng will obviously increase the usage of CPU.
- Allocating memory space by using `make()` will cause uncertain footprint. And GC may be triggered under certain situation.

## Unresolved questions

- The program itself takes about 68MB
- It takes time to allocate big memory (over a giga bytes) without the parameter `--time`. For example, it will take 3.5s to allocate 12GB memory space.