Next? level documentation

[ruffrecords]

I am a hardware engineer with more than 50 years of microprocessor/microelectronic based development under my belt. Back when I started there were no micro controller software engineers. We hardware engineers did all the coding. So I have written lots of code in many assemblers, Forth, Basic, Pascal, Tcl/Tk and C, most of it for real time applications. For me, MicroPython is a true revelation. A high level language that gives you fingertip access to the hardware from the keyboard, without the intervention of an enormously complex IDE, makes it capable of cutting development time and effort in half.

Add to this the ready availability of very cheap microcontrollers with unbelievable quantities of ROM and RAM coupled to powerful peripherals, and embedded system development ought to be a breeze.

But I am struggling the grok Micropython itself. I have a hardware engineer's detailed bottom up view of microcontroller applications and Micropython is clearly written by extremely capable software engineers with a predominantly top down view which is reflected in the documentation (as you would expect because after all Micropython IS a high level language).

But where I struggle is in understanding how best to implement the middle ground, where the nitty gritty hard real time stuff takes place. Even simple things like how to efficiently pass information to/from the foreground and background. I have done a lot of searching and, for example, after frightening myself when I discovered integer variables are immutable ???? I came across small integers and a few more details of the inner workings of Micropython that I feel are essential to being confident I can write code that will work. I also came across quite few computer science terms that I am not familiar. I am used to passing by reference or by values, but it seems you can also pass by assignment (whatever that is) and pass a reference by value.

So, what I am looking for is a more detailed explanation of the lower level aspects of Micropython. I am happy to continue searching and build up my knowledge piecemeal but I just wonder if I am missing a great resource that already covers this.

Cheers

Ian

[ruffrecords]

To partly answer my own question I came across a couple of documents entitled the GraftSense Performance Optimization Guide and a similar one for memory optimisation, both in github. Unfortunately they are in Chinese. Does anyone have an English translation of them?

Cheers

Ian

[osodrac-lab]

Micropython is (almost) like Python and there are tons of tutorials all over the place at all levels for Python. Most of them will be relevant to Micropython and will get you started. Enjoyably.

[scruss]

To partly answer my own question I came across a couple of documents entitled the GraftSense Performance Optimization Guide and a similar one for memory optimisation, both in github. Unfortunately they are in Chinese. Does anyone have an English translation of them?

Google Translate often surprises me with its usefulness. Give it a shot.

As for the low level on ESP32, it's mostly down to the Espressif documentation around ESP-IDF for the particular chip that you're using.

[peterhinch]

I discovered integer variables are immutable ????

You discovered wrong. Strings are immutable. Integer (and floating point) variables are mutable.

In my opinion it is worth learning the Python language at a PC with a tutorial before making the transition to microcontrollers.

[peterhinch]

Yes, this confuses many beginners. To clarify, consider this:

x = 42
def foo():
    x = 23

If you run foo() a new local integer x is created and the value 23 is assigned to it. The outermost x in global scope is unchanged at 42.

To fix this, you would have

x = 42
def foo():
    global x  # Note that global is declared in the function body
    x = 23

That version of foo would update the x in the outermost scope (from 42 to 23).

The contents of the physical storage location of that variable would be updated: integers are most definitely mutable. However they are passed by value: in that instance a copy is made. As @robert-hh says, it's important to learn the Python scoping rules.

[beetlegigg]

I got a bit confused with the comment on physical storage location and passing by value comments but after a check of the id of the outer and inner x variables I think I see what you are meaning. I did this in python, but I presume that micropython would be the same.

x = 42
print('outer x id:', id(x))

def foo():
    global x  # Note that global is declared in the function body
    x = 23
    print('inner x id',id(x))

foo()

print('revised value of outer x:',x)
print('outer x id:', id(x))

result:
outer x id: 4319290656
inner x id 4319290048
revised value of outer x: 23
outer x id: 4319290048

So when the function created a new value for x, it created a new memory location and made the original global x variable point to the new memory address. Hence I guess thats what you mean by 'However they are passed by value: in that instance a copy is made.'

It seems to depend on how 'mutable' is defined. The value can be changed, but the original memory location is discarded in favour of a new location. I leaned something new in my coffee break :-) though I don't think it will have implication for my python programs.

[peterhinch]

A few comments:

• I'm surprised by this.
• It can be replicated in MicroPython.
• It has no consequences for normal code. I've written and studied code using hard ISR's and threading (probably the most challenging cases) and it always behaves as if integer variables are mutable.
• The function foo above would work as a hard ISR. It is not allowed to allocate RAM in a hard ISR. If writing to a global causes it to change address, why is no exception raised in this case? @robert-hh ?

[sosi-deadeye]

You discovered wrong. Strings are immutable. Integer (and floating point) variables are mutable.

In CPython int, float, complex, str, tuple, frozenset, frozendict and some other types, are immutable, so you can't change the underlying value(s).

It should be the same with Micropython, except there is an optimization.

But, id() and hash() are not the same, as CPythons id() and hash() functions. Micropython has many optimizations and also constraints in memory. So Micropython can't just cache values from -5 to 256, because RAM is very limited. Instead, the hash of the values is the value itself. Also the id() of int (I tested only with int) is after arestart the same. So I guess the implementation does not randomize the id of integers.

[robert-hh]

why is no exception raised in this case?

When you run the example in MicroPython, the values are different. In MicroPython, the id of a small int is value * 2 + 1, because the value is stored in the object. A small int does not have an ID the way e.g. a bytearray has.

[fojie]

Hi all,

I have realized that there remains a lot to be learned about this topic – at least for me. I suggest to study this carefully:
https://realpython.com/python-mutable-vs-immutable-types/

/Folke

[ruffrecords]

Just as an aside here is a recent REPL conversation:

aa = 42
id(aa)
85
aa = aa + 1
id(aa)
87
aa = aa + 1
id(aa)
89
print(id(aa))
89

Why is the id changing? What exactly is the id in micropython?

According to my Python Pocket Reference it is "id(object) returns the identity integer of object which is unique for the calling process among all existing objects (i.e. its address in memory)"

Cheers

Ian

[ruffrecords]

And here is an extract from the mutable and immutable types section of the Real Python web site:

"An object’s value is probably the only characteristic that you’d want to change in your code. An object that allows you to change its values without changing its identity is a mutable object. The changes that you can perform on a mutable object’s value are known as mutations.

In contrast, an object that doesn’t allow changes in its value is an immutable object. You have no way to perform mutations on this kind of object. You just have the option of creating a new object of the same type but with a different value, which you can do through a new assignment.

In Python, you’ll have several built-in types at your disposal. Most of them are immutable, and a few are mutable. Single-item data types, such as integers, floats, complex numbers, and Booleans, are always immutable. So, you have no way to change the value of these types. You just have the option of creating a new object with a new value and throwing away the old one."

Cheers

Ian

[robert-hh]

That is more an implementation view and seems confusing. To someone writing Python code variables look mutable, meaning that one can assign different values to these. One can write x=27;x=42;s="Hello";s="World". So the symbol can be linked with a different value. The Python runtime implements it's method to deal with these value assignment. The value is obviously stored someplace in memory. The Python runtime may be set-up to chose a new memory object for storing the value, or it may use the same place again. But these are opaque implementation details, and most of the time not relevant for Python code.
Micropython has 4 types of object storage, detailed in py/mpconfig.h. Depending on the data type and it's value, the value of an object is either stored & replaced in the object itself, or the object holds a pointer to a different place in memory where the value is stored. If the value can be stored in the object, assigning a new value does not allocate memory. In that case it works fine in a MicroPython hard IRQ handler. An example are small integers (and boolean), which are stored in the object. Small depends on the object representation model and the word_size of a machine. For most MicroPython ports small means a value that fits into word_size -1 bits.

[ruffrecords]

I agree it is more of an implementation view but as a hardware engineer that is exactly the (next) level of understanding I want.

Cheers

Ian

[scruss]

Why is the id changing? What exactly is the id in micropython?

It also changes in CPython:

Python 3.13.7 (main, Mar  3 2026, 12:19:54) [GCC 15.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> aa=1
>>> id(aa)
10820304
>>> aa+=1
>>> aa
2
>>> id(aa)
10820336
>>> bb=aa
>>> id(aa)
10820336
>>> id(bb)
10820336
>>> aa is bb
True
>>> bb+=1
>>> id(aa)
10820336
>>> id(bb)
10820368
>>> aa is bb
False

[robert-hh]

According to my Python Pocket Reference it is "id(object) returns the identity integer of object which is unique for the calling process among all existing objects (i.e. its address in memory)"

Still different objects may have the same ID. But one can rely on the fact, that objects with the same ID have the same value. How the ID of an object is created depends on the implementation of the Python runtime.

[peterhinch]

A fascinating and informative discussion. The practical take-aways seem to be:

• id is implementation dependent and does not necessarily correspond to a memory address. (In contradiction to some online docs).
• In a Python context the meaning of "immutable" can be technical and non-intuitive.
• In practical coding and in its generally accepted sense integers can be considered mutable. This applies even in "advanced" scenarios such as interrupt handlers and threaded code.

The fact that strings are immutable is worth bearing in mind when considering RAM usage. Consider:

a = "abcd"
a += "e"

While this works, the resultant "abcde" is a new string. The garbage collector will eventually recover "abcd". This can matter if repeatedly making small changes to big strings.

[ruffrecords]

Thank you everyone for the excellent replies. It is indeed a fascinating topic.

My main take away from this is that indeed everything in MicroPython is an Object, even numbers.

So if I write x = 1 then x is a reference to the object 1 and when I write id(x) I get the id of the Object

If I then write x = x + 1 I get a new reference to the new object 2 and it will have a different id

Although x now refers to the new object, if I assign a different variable to Object 1 it will have the same id as x did when it referenced Object 1. So if I write y = 1 its id will be the same as x was when I wrote x = 1

For me the key concept is that variable assignment just instantiates an object and returns you a reference to it.

The object has values. If you can change the values (of that instantiation of that object) then the object is mutable. If you cannot then it is immutable. If it is immutable, the the only way to change the value represented by your variable is to instantiate a new one with the required new value.

My only remaining question is for how long do unused objects persist? Are they subject to garbage collection.

The other thing that occurs to me is that to say integers are (per se) immutable is inaccurate. The integers in an array are certainly mutable but single integers are not.

Edit: I just discovered the above is not true for floats. Separately instantiating two variables with the same value results in two different id's. However if you write r = s = 3.14 they will have identical id's. Hmmmm

Cheers

Ian

[peterhinch]

Calling the ID a "reference" to an object is misleading. When you call by reference you can access the object in the outer scope through the reference. By contrast id(obj) produces an opaque value which cannot be assumed to provide a means of accessing the object.

[ruffrecords]

[pedant] I think I did not actually call an ID a reference[/pedant]

I think I said a variable references an object. The object has an ID. Two variables referencing the same object will return the same ID

geekforgeeks.org definition of ID is (my emphasis):

"The id() function is used to get the unique identity number of an object in memory. This identity remains the same as long as the object exists. If two variables have the same id(), it means they are pointing to the same object in memory."

Edit: Here is a link to the above: https://www.geeksforgeeks.org/python/id-function-python/

Cheers

Ian

[beetlegigg]

As I've been following this thread I will add my tuppence worth.
Real Python says:
"
The built-in id() function returns the identity of an object, which is a unique and constant integer that identifies the object during its lifetime.

In CPython, this identity corresponds to the memory address where the object resides:
"
But that leaves it open for other python implementations to have an id thats not related to a memory address at all. And as explained by robert-hh in micropython, at least for the smaller integers falling with a 1 word storage, the id is simply the number value x 2 +1 and nowt to do with the memory location. So an integer of 27 would get an id of 55

But of course wherever in memory the 27 is stored the id 55 represents that memory location, but we just dont know where (and dont really want to know :) )

[ruffrecords]

I tend to agree. We seem to have strayed somewhat off topic. The discussion of "The Meaning of ID" only became about because I found it to be a useful tool in determining the unique object associated with a given variable, and that was of interest only in determining if said variable was mutable or not.

Edit: so to to get back on topic here is the last line of my original post: "So, what I am looking for is a more detailed explanation of the lower level aspects of Micropython. I am happy to continue searching and build up my knowledge piecemeal but I just wonder if I am missing a great resource that already covers this."

Cheers

Ian