This year I have decided to dedicate some (more) time to improving the MPS “ecosystem”. I have already been able to do some of the needed work thanks to my current long-term client but I feel that this is not enough and much more can be done in two areas: speeding up the uptake of newer MPS versions by large “platform” libraries, and improving their modularization.
I think these problems both have a common cause: inadequate build tooling. So, in 2024 I intend to spend roughly only half of my time doing paid work and the other half on improving the tools.
In the rest of this email I want to describe the ideas in a bit more detail.
Proper continuous integration of platforms
I want to set up proper continuous integration of MPS with mbeddr platform and IETS3 (KernelF). The main/master branch of each library should be regularly built against the latest successful build of MPS master branch. If we can build them regularly, it should be easier to pinpoint compatibility problems because the delta from the previous working build will be smaller.
In order to do that, I have already created a build configuration in the mbeddr TeamCity to transfer
the latest build of MPS to a Maven repository where it can be more easily picked up by the build scripts of the
libraries. I have also arranged for a special repository called maven-mps-prereleases to be
created in the itemis Nexus where the TeamCity builds will be placed. This repository is a hybrid between releases
and snapshots: uploaded artifacts have fixed version numbers but will be regularly cleaned up.
The next step would be to modify the Gradle builds of the libraries to build against these prereleases.
Tightening the integration checks
While simply building a project against the latest MPS version will show some problems, it is also necessary to do more automated checks. The CI builds should also check the projects for any deprecated code, code that wasn’t migrated to the latest version of a language (this is likely to happen due to cascading merges) or other violations. Daniel Ratiu’s mps-qa library provides “linters” with most of these checks, as well as whitelisting mechanisms in order to improve the quality gradually, and I am going to look into adding these linters as part of the build for each library.
Incremental builds
As more and more checks will be added to the build, it is going to become slower. In order to mitigate this, we will have to look into making the build incremental, so that only affected parts are rebuilt. (Of course, a new MPS version will likely mean that the entire library has to be rebuilt, but a change in one component of a platform should not require rebuilding and reanalyzing unrelated components.)
I have already had some success and gained some experience with incremental builds in a client project but adding
incrementality requires providing more information about the dependencies (inputs and outputs) of a particular
component from MPS to the build tool. MPS already keeps project information in two places: the .mps directory and
the MPS build scripts, and we would be adding a third, so this needs some thinking and automation.
Modularizing the platform libraries
I hope that by improving the tooling around incremental builds, it will also be possible to split the current MPS platforms into smaller components. It is currently impossible, for example, to use just the assessments framework from the mbeddr platform without bringing along most of the rest of the platform. My inspiration here is Spring Framework which is releasing multiple libraries that can be used together but do not have to.
The biggest obstacle to achieving this currently is that this would mean lots of small build scripts, both in MPS and in Gradle, and I would like to avoid having to keep them all in sync. This may also need some changes in the way MPS build scripts describe dependencies (as they cannot handle transitive dependencies very well, for example).
You can help
If you find any of this interesting and would be willing to join me in this effort, let me know.
If your company wants to sponsor any of the above in order to have the benefits materialize sooner, I would also be open to that.
Did I miss something? Do you think something else is more important than these issues? Hit Reply and let me know.