Python What are the limitations of Codon, the new Python compiler?

[jim mcnamara]

TL;DR

Python is largely an interpreted language, Codon compiles Python into fast executable images.

See this story here in a press release from MIT:
https://news.mit.edu/2023/codon-python-based-compiler-achieve-orders-magnitude-speedups-0314

Basically, any working Python module can be compiled one time only, and then run 10+ times faster than interpreted code, with no further changes to correct tested code you have already available. There is a Github link in the article.

 

[jedishrfu]

This work is long overdue for Python and I'm glad its being addressed. However, there are some caveats listed in their paper conclusion mainly to do with the dynamic features of python.

One other issue with python is the need to pull in so much extra stuff to run your code. Sadly, it's very difficult to trim your code down to a minimal level. I ran into this issue while doing docker images.

https://groups.csail.mit.edu/commit/papers/2023/cc_Codon.pdf

6 Conclusion

We have introduced Codon, a domain-configurable framework for designing and rapidly implementing Pythonic DSLs. By applying a specialized type checking algorithm and novel bidirectional IR, Codon enables easy optimization of dynamic code in a variety of domains. Codon DSLs achieve considerable performance benefits over Python and can match C/C++ performance without compromising high-level simplicity. We note that Codon is already being used commercially in quantitative finance and bioinformatics.

Currently, there are several Python features that Codon does not support. They mainly consist of runtime polymorphism, runtime reflection and type manipulation (e.g., dynamic method table modification, dynamic addition of class members, metaclasses, and class decorators). There are also gaps in the standard Python library coverage. While Codon ships with Python interoperability as a workaround to some of these limitations, future work is planned to expand the amount of Pythonic code immediately compatible with the framework by adding features such as runtime polymorphism and by implementing better interoperability with the existing Python libraries. Finally, we plan to increase the standard library coverage, as well as extend syntax configurability for custom DSLs.