Unity Packages: From WTF to FTW With Needle Tools, by Felix Herbst

The first game I created was as a student in a small team. Incredibly over-ambitious, we immediately started with a multiplayer mobile game series, of course, as the first thing to try as students. And that actually worked pretty well. We got super far. We had multiple little games that were actually networked and doing things with each other. So it went surprisingly well for how ambitious it was.

Yeah, thanks. I'm Felix, and today I'm going to talk about Unity packages and our path at Needle from trying to figure out what these new things are that have popped up in the editor at random points in time, to building entire projects around them and leveraging them to their fullest. It's an interesting topic I wanted to talk about for a long time because there's so much confusion and misunderstandings, even in teams at Unity and teams using Unity, on what these packages are good for and not good for and how to use them. So I hope you can uncover some of that today. It's going to be quite a wide talk; I have a lot of topics I want to cover, so we're going to get started.

First, a little bit of overview. Besides running Needle, I'm also running a brand agency, Prefrontal Cortex. This is where a lot of the need for packages first arose. We saw them appear in the editor in Unity 2018 as a new way to manage and structure code, and immediately jumped on that as a solution for copy-pasting things between projects, which was pretty much the standard at the time for reusing code, either that or submodules. Both have their problems. By now, both Needle and Prefrontal Cortex have their own package registries, similar to what you're using when you're consuming packages from Unity. There are about 40 on the PFC registry, about 23 on the Needle registry. For comparison, there are like 250 plus packages on Unity's registry, at least the ones that are public. A lot of people are by now aware of OpenUPM, which is another registry for hosting open source stuff, where you have like 1800 plus packages by now from various people. So that's just an overview. Everything at our companies is now structured around packages. There's no way we're copy-pasting things from one project to another anymore.

To show how that all means and how small teams can manage packages in this way, we're going to do a little bit of a refresher, starting from scratch from what packages are, and then gradually going into more complicated levels on what packages can do and how they work.

On a very basic level, what has changed for a lot of people is a new folder appeared at some point. In Unity 2017, if you have an empty project, it looked like this: you have your scene, you have your stuff, everything's there. In 2018, that changed. Unity started modularizing some of their core and putting things into packages. That's how a lot of people noticed it first: there's now an Assets folder and a Packages folder. That was about it at that point in time. Not a lot of documentation. This has changed a bit, but still there's lots of untapped potential I want to cover here. From then, there have been a couple of design changes, and nowadays this is what you see. By default, there are more packages, so more of Unity's code has actually become modular and accessible in this way. The package manager has also changed.

That brings us to kind of level two: we have packages, how do we manage them? There's a package manager by Unity. Initially it looked like this, gradually got more refined, got more features. A lot of you know this, so I'm not going to cover it a lot. One recent interesting addition is features, so you can have feature sets, basically packages that are in themselves pulling in more other packages, and you can make your own. This is another interesting thing for studios and larger organizations that want to always bring a number of packages into a project. There are more changes, like this is fresh 2023.1 where you can see a little bit more about which packages have dependencies. I'm going to cover this a bit more in later slides. That's our package manager. It's relatively minimal and it's clearly designed for consuming packages, basically for installing them from Unity's registry.

How does this look for a specific project? You always have a project manifest. In every Unity project nowadays, there is a Packages folder. In that, you have a manifest.json, and in that manifest.json you have the identifiers and versions of packages that you're using. That's not the only thing there. There's also a package lock, which is pretty important. Please make sure you have this as well in source control. I've seen companies exclude that and then wonder why things break when Unity tries to make sense of all the packages in your project. It resolves which versions can coexist at the same time or which version of a git package to pull in, and this is all stored in that package lock. So it's important, please keep it.

One interesting concept of packages is the idea that by default they're immutable. When you download a package from Unity's registry, you can't change it. You can open those files, look at it, read it, but you can't start changing it right away because the system will notice and reset that package. The idea is you can bring in packages and you have a given state. But one great thing about packages is that you can make them mutable, you can make them editable and adjust for your own use cases. One thing where people do that a lot is, for example, the Scriptable Render Pipelines, where you can take Universal Render Pipeline and make it your own in a specific project. I'm going to show a quick overview of how the process looks when you want to actually edit a specific package, because there's also a lot of misinformation around that.

On the most basic level, you have your packages, each of these are immutable. If we for example now want to change something about the Timeline package here, we can open it up in the Explorer. This actually brings us into the project Library. Each project has a library where all the stuff is cached, including packages that you have pulled from registry. What we can do here is we can actually simply grab this version of Timeline here, go some folders out, go into Packages, and drop that here again. Just by doing that, if I focus Unity again, Package Manager is going to pick that up and see, hey, there's a new version of this Timeline package that now is mutable. Some of our tooling that I'm going to share later is displaying that here. So we see these are all still immutable and this one is now actually editable. I could go in here and actually change how Timeline works if I have a specific project that requires doing these adjustments. On a basic level, this is a core difference here between packages that you can't edit and don't want to edit, and the point in time where you decide I now want to start doing changes to a specific package. You can go both ways. You can also make a package immutable later on by publishing it again or packing it again. I'm going to cover that in a moment.

So we saw a little bit of how packages look in Unity, how you start editing them if you want to. We're also going to cover what if we want to make our own packages, because obviously that's one of the most interesting aspects of the system. It allows companies to structure their projects differently and to start putting all of the code and where that is in various places into reusable pieces. On the most basic level, what we have on the left here is a complete package. All we need is one folder and one package.json, and then we have our own. We're going to do that here. We're going to go back into our project, heading into this Packages folder. We already have our Timeline, and we say we want to make a new one, like my package. All we need here is basically one package.json. We need a name and a version. There are specific rules how package names and versions work in Unity, which differs a little bit from the parent of that, the Node Package Manager system. For example, you have versions always in this three-part system and not in scopes. But that is not that important here right now. On a very basic level, we now have our own package. All it has is this one file here. It doesn't look nice, so we're going to adjust its display name here.

This is already, on a super basic flow, how you make your own. Of course, if you're doing that in production, there's a little bit more to it. You will want to have more info in the package.json, because that helps people to see what it does, to see who the author is, the description, and all kinds of things that Package Manager can also show back, and even including samples that people can install for your tooling if you put it correctly into the package.json.

As you already saw, my packages in Unity look a little bit different than they do by default, because at some point we started building our own tools for making common operations with packages easier. Common operations means things like I want to have an understanding of which packages in my project are from which version, or which I can edit and cannot edit. That becomes more and more important the more you compose your projects from packages. You can actually install these on your own. The code is not open source yet, we're planning to do that at some point in the future, but the package is already available on our package registry. I already have that in the project here. One obvious change that this does is it displays which of the packages in your project are editable and which are not. What it also does is in Package Manager, it shows the sources of packages better.

By default, if you have Package Manager open, you don't quite see which packages come from where. So let me just quickly turn that off. This is how Package Manager looks by default. You basically have the versions, you have some state about them, you have at least the author, but you don't really see where this is coming from, which of these are actually part of my code that I want to be able to edit. This is what our extensions change. They add these tags here, for example: this package comes from git, this package comes from the Needle registry, the one we just created is an embedded package, same as this Timeline here. What it does, and that is for us the main factor for actually being able to maintain our packages and our code well, it adds stuff on the right side here for managing the details of a package, being able to quickly jump into the package.json or editing the changelog, or tools for publishing and for verification. This is all stuff that I'm not going to go into super deep detail here. You can try it all later on your own. But the gist of it is it makes it super easy to see if you did everything right with your packages. For example, you should always have a changelog that matches the actual version you're releasing, which is something that even Unity's packages sometimes get wrong.

From this point, I think it's worth looking a bit more into how this package system works under the hood, because there's a lot of interesting details in that for how you can work with it and modify how things work. On a super basic level, in each editor installation you will have a Resources Package Manager folder which contains a bunch of things related to packages. There are, for example, the packages that already come bundled with the editor. For those of you who have been wondering why Unity editor installations grow exponentially over time, it's because Unity starts adding more and more packages that they're shipping with the editor to keep download times low for users that open things for the first time. For example, you have one gigabyte of three different versions of Burst in each editor installation nowadays because you may need them. Obviously all of these can be fetched from Unity's registry as well, but it seems it was easier to ship them with the editor. What this shows us, and which is an interesting detail, is that you can ship packages as archive files. This is a .tgz, an entire gzip file, and it contains an entire package. But you can reference this as something immutable again. Even when it's one file, you can reference this into projects and you can make your own.

One folder inside Package Manager is Project Templates. Actually, the templates that you can install from the Unity Hub are packages in themselves that kind of get unpacked to projects. You can also modify these and make your own templates, which is something that people have been waiting for from Unity to make happen, but actually you can already do it. A lot of companies are doing it for their internal workflows for specific setups, bringing their own packages in. This is how you can make a package archive and have it be a template for future projects.

Another detail here is how does Unity actually manage which package versions get shown if you are in the editor and you list what packages are actually on Unity's registry which you can install. Where does this list come from and how does Unity know which packages to show in which version? This is where this fancy file here comes into play. This is on Unity's registry, it's kind of a special package that contains metadata for these packages. These are available on the registry, some are searchable, some are not searchable because they're always installed as dependencies. They make continuously new versions of this. These are kind of the entry points for packages that can be searched. Matching to this, each editor installation contains another list of how you map to that and which versions to show for that editor installation.

One thing that's interesting here is that by looking at this list, one can programmatically get a pretty good list of which packages Unity actually has available. One thing we did relatively early on at Needle is making sure that we can look at the various versions of packages better based on this list. What happens when you install a new version from Package Manager in Unity is it just switches that code on disk to get a new version of the package. Say you're updating your URP from one version to the next, you get something new, you don't really know what has changed and why did Unity update that. There must have been a reason. Usually there's a tiny changelog and something like this, and this is not enough if you're actually working on production projects where you need to modify these packages but still be able to follow if Unity does bug fixes. Say you are URP and you want to modify this and bring your changes back. So we started this project Mirror, where basically what we're doing is we're looking at this file regularly and check, do we know all these packages, have new versions appeared for these packages? If yes, then we're mirroring them to GitHub.

How that looks is that you have an entire GitHub account that's responsible for keeping track of versions of packages. Basically, each and every package that Unity releases on the registry gets mirrored here automatically. From this, you can dive deeper into the changes that have been done to them. One example is the Addressables package that a lot of people are using and modifying in their projects. If you're just using the one from Unity, then basically Package Manager gives you that new blob, like here's a new version of Addressables. Having all of this as versions on GitHub, in GitHub's system of comparing things, looking at changes, opens up this interesting capability of, for example, comparing between versions. I can jump into this version here and see the changes that have been done. This is all just mirrored from what Unity puts in their changelog. We see like 5th of October was the last update here, we have a specific changelog. But now with Mirror, we can actually go in and see what did actually change between these versions. This is super helpful for keeping track of things changing in Timeline or in URP or in any package where we've seen a lot of projects that go deep on working with these packages and modifying them for their own needs. You need to be able to see what has actually changed from version to version to not lose track of that.

We have a bunch of changelog stuff, but then we're getting into changes only they have: built target changes, or things have changed how errors work, different calculations. Usually you shouldn't have to worry about this, and unfortunately this is still how Unity sees that: you shouldn't have to worry about these things, it's an internal package, you should just consume it. But that's not how reality works for a lot of productions where you need to be able to see what happens and why and adjust it. Often the flow for this is something like you can dive into specific code in a package via a Rider and look at what happens where, and then you can actually check out on Mirror why specific changes have been made or why your code broke now due to a change in Unity's packages. This is a very nice learning resource, but it's also a crucial part of these modification workflows. What we earlier did on just copying in the Timeline package is okay if you want to do a quick modification, but it's not great if you want to actually do this in a production where you later may want to update to a newer version of Timeline. What a lot of people are doing is they're forking from Mirror actually. The Addressables package alone has been forked 439 times from here, people doing their modifications because they want to be able to get the upstream changes from Unity at some point.

Mirror makes things easier for modifying Unity core packages. Actually, one thing I forgot is we do have a Mirror integration in our package tools. If I go into the Input System here, there's a direct integration for saying, let's see what has changed here, let's directly compare how the Input System changed between these two versions. So basically after an update, if something is broken, you can take a look at what has actually happened.

This brings us to kind of the next level. We now have a better understanding of what packages are, how to modify them, how to move them around. If you're doing this for an organization, you want to be able to publish these packages so that other people can download them again. The way this works is always through package registries. Basically, if you're interacting with Package Manager, all that happens is that it lists which packages come from specific registries. So like this is what comes from Unity's registry, and there's another field here for My Registries where we now have all the packages that Needle has online that are compatible with the specific Unity version. The way this works is that you can have many registries in one project. It's quite common that you're consuming things from your own registry, from OpenUPM, from Unity, and potentially a couple more, and have specific packages from each of them that you're composing your project of. Package Manager allows managing this here. It actually all lands in the package manifest file. Everything here is actually something that you can do programmatically in larger pipelines.

How it looks in the manifest is you have these scoped registries where you have a name and URL and scopes. Same thing as in the UI, but actually something that you can modify through tools. For example, you can just write this manifest file from a tool and say I want to set up a new registry here, or give this to people as a template for starting projects in your organization that should use specific packages. There are a number of registries that you can basically readily use. There's OpenUPM which I mentioned before, an open source registry that now has like 1500 or more packages. What's often more interesting for organizations is being able to host your own registry. For example, we are using Verdaccio, which is actually an npm, not directly related to Unity, it's more like a Node.js code registry that you can use and set up your own. This is super easy to get started with. It's as simple as running your own on Glitch. Glitch is a super useful web service for having tiny Node.js applications. With just one click here, we can actually remix our own package registry, ready to be used.

Let's just have it run for a moment here. This is also something that we built to make it easier for people to get started with packages and to experiment with registries and see that flow. Of course, later you should definitely put this on your own infrastructure or onto something bigger. This is a super tiny instance, I wouldn't use this for a large organization, but for experimentation it's great. Basically, our registry is now already live, our new one just remixed here, which looks like this. From here, we could basically add that in Unity, publish packages to it, send it to other people, and they can install packages from there. So great experimentation ground. I'll keep these links on the slides of course, so you can later experiment with this on your own.

If you have your own registry, then of course you need to be able to also publish packages to it. The usual way for doing this is through the command line. You're able to, for example, open up this one in the command line and then there's an npm command for pushing that to a registry. There are some helpers for that here as well for publishing. For example, our little package here doesn't have a changelog yet, so we can click this button, now we have a changelog, and we can make sure to fill that out for what we have done with our package here. Hey, we have a changelog now. Should also have the right version. There are more conditions to this, but ultimately you will open a command line then and do something like npm publish to publish it to a registry that you have set up. We won't do this here, but I wanted to point that out. This is one flow how you can do this. If you're working with OpenUPM or other registries, there are actually GitHub Actions and things like this where you just push to a specific branch or you tag a release and OpenUPM is automatically going to fetch that and make a new release. Multiple approaches to this, but it's relatively simple and well documented, so I'm not going to cover it more here.

One other piece is how do people now install these? We already saw one approach for installing packages via manually editing details in the package manifest, but there are other ways on acquiring packages and putting them into your projects. One, as shown earlier, is when you want to get this from OpenUPM. There is a way to manually install this where you, for example, add the right scope. We can do that quickly here. We can say we want OpenUPM, we want specific URL here, and we want a specific scope. This is enough to basically tell Package Manager where to look for these packages. If we open up Package Manager and go on My Registries, and if we didn't mess anything up, it should be able to pick that up. Why doesn't it? Oh, because we have a super wide scope. That's actually good that this happens. One of the pitfalls when you work with packages is that you have to be careful about these scopes. For example, now our Needle registry here kind of has a wider scope, we want to have all Needle packages, then OpenUPM here where we just want to have a specific one. It's one of the cases where you can't fix this in the Package Manager UI, but you can fix it in the manifest. We're going to do that. Basically, we just need to switch these two entries and tell Package Manager: please first look on OpenUPM and then secondly you can look on the Needle registry.

There we go. Now you see that basically Compilation Visualizer here comes from a different registry, which is again something that you wouldn't see in regular Package Manager, but you can see it through our extensions. Now that we have this set up, we can install that and have our new package from a new registry in our project. It doesn't actually matter if this registry is the Unity registry or OpenUPM or somewhere else, the flow is always the same. Now we have this here as an immutable package because we installed it from a registry.

That's the OpenUPM flow. You can also install packages from git directly via git URLs, for example Unity GLTF or something else. I wouldn't generally recommend that when working with packages because of these earlier mentioned problems with package resolution. Basically, when you're using a git URL, you're always getting the version that's currently live on that git repository, at least if you don't specify more information in the path. You can do that, I'm not going to go into details on it. I would always recommend if there's a registry version available for a package, do that.

One more way to install things is as a hybrid package, and it's something I want to briefly cover as well. Hybrid packages are an interesting middle ground between the old world of Unity packages that you still have on the Asset Store, for example, and the new world of packages on a registry or UPM packages. What this does is you have a Unity package, but what it actually contains is a package. Unity packages are just zip files essentially that unpack into your projects. We found a way to modify them. By default, Unity packages only allow you to unpack things into Assets, but we found a way to modify that and to actually tell it: hey, this is where I want to unpack, I want to unpack into the Packages folder. What that does is when we're importing this, we're getting a package here that is now mutable and that we just installed from a Unity package. This is something that we coined hybrid packages. I think I have a few more slides on this later on. So one more way to install things, and especially useful for the Asset Store or for other cases where there's no real integration of a registry yet, but you still want to be able to work with packages.

Besides this manual workflow of entering stuff in the package manifest or in Package Manager, there's also a package installer service that we've built years ago now, where you can just construct a URL and tell it to please give me a Unity package that will set up a package in my project. As an example, we can again download Compilation Visualizer here. Let's start Package Tools. This one works. What this does is also a Unity package, but you have an installer. This installer is doing slightly more things than just copying this in your project. It actually sets up the registry. Package Manager complains and says, oh no complaints, but tells you hey, there's a new registry that has been added by someone else, which is what the installer has been doing. Then if we wait a few moments, we should have this here as well. There we go. This basically installed the Package Tools by setting them up as a package from a custom registry, adding that to the manifest, and then the installer removes itself. We built this because, as mentioned, you basically always want to have registry packages, you don't want to just copy things around. But Unity makes the flow too slow for mass usage of packages across projects. Always having to manually do this for new projects is just cumbersome, which is why we have this installer, which is free and open source. A lot of people are using it. I think we have some statistics that something like 50,000 package installations have been done over that installer in the last years.

One thing that's relevant especially to organizations is packages sound nice, but then you realize you don't want everyone to be able to access these packages. The interesting part is since this is built on top of npm, like Node packages, authentication is something that you can do. People have been building packages to do that for you, to basically manage credentials for registries. There's a really cool one from Heloti that allows to manage these credentials, and we have a simplified version on our own registry free to use for people for managing package credentials. I can again show how this looks. Basically, if we open our Project Settings, we have a new field Credentials here where you can add credentials for a new registry. You can log in, enter your password, and so on. This makes it possible to use packages as well in corporate environments where you need to protect the stuff. You may have it behind a VPN, you may have it on a public URL like Unity packages work or like Needle's packages work, but some of them you want to protect and only make accessible via authentication. Contrary to what the Package Manager documentation from Unity tells you, it's super easy if you use these kind of helper packages here, which behind the scenes do the right setup so that Node.js Package Manager knows which credentials to fetch and what to pass to the server. Actually a solved problem for quite a while, and a number of companies are using this.

I already mentioned hybrid packages quickly from installing them. As a repeat part of that, hybrid packages are a bridge between the old world of Unity packages that are just basically zip files, to new UPM packages that can do more. One thing we realized while working in both of these worlds is that Unity packages can unpack into any folder structure if you bend them a little bit, and UPM packages only require a specific folder structure. This is also something that we saw a lot of companies adopt for various cases, and that you can even ship things to the Asset Store. This is the way if you nowadays want to ship actual UPM packages to the Asset Store, and you can do that through that hybrid packages tooling. In fact, recently Unity released new versions of their Asset Store tools that contain this as kind of a hidden functionality that you can activate with a keyword. Basically, we had a lot of discussions, I told them about our approach here and that we found a way to bend the will of the Asset Store to allow packages sooner rather than later. You can do that. Some of the Needle packages on the Asset Store are shipping in this format, which is kind of a compromise between this old way of Asset Store things just randomly unpacking everywhere in your project and the new way of structuring things and modularizing them.

With this, we're getting into some of the more complex topics. We only have a few minutes left, so I'm going to show most of this right in Unity on how this looks in practice for more complex projects and how do you actually work with packages if you have all these setups. For that, we're going to jump into what I'm calling a package playground. Basically, it's one project that contains all our packages. From here, we do testing. We test that various packages nicely interact with each other or that they are ready to be used across different Unity versions. One crucial aspect if you're actually going into developing packages and running your own registries is you should make sure that this is properly tested. Otherwise, people update something from Package Manager and things are broken. You want to always make sure your packages interact nicely with each other, and you want to make sure that they interact nicely with various Unity versions. I know that in the theories that Unity usually has about how people are using their editor, everyone has this one project of their game that they're working on for three years and they're never changing the Unity version. This shouldn't be necessary in practice. We've seen companies that have hundreds of Unity projects across all versions. Prefrontal Cortex, like my agency, is one example for that. Projects are shorter term, it's not five-year-long games, it's working on something for weeks or months and then always starting with a relatively fresh Unity version. There it's really important that packages work across various versions of Unity.

How that looks is basically that you have multiple Unity projects. All of these here are Unity projects. We're usually putting the version right in the folder name so that we know what is what. Each of these projects references the same packages. This is pretty cool because it means that you can have multiple instances of Unity open that all use the same underlying package code. The way this works is that in your manifest, you can actually have file references. Basically, instead of referencing packages from a registry or just locally, you can tell it: hey, this package comes from a specific place on disk, use that. That immediately allows you to compose projects of the same packages. It's not a cache, it's not the same as what Unity does with their library on copying an immutable package into it. It's actually the version of these files on disk. What that allows to do is basically having these multiple versions open. Let me just open a few more here. Here we go. You can basically have multiple instances of Unity open, multiple instances of Rider if you must, and working on the same code on disk across these various projects. That makes things that are traditionally very hard suddenly easy. For example, adding the right #if defs to your projects to ensure that they work across versions, or ensuring that a specific package works with one version of your URP and another, all these kind of things suddenly become a lot easier to maintain because you're effectively working simultaneously on all of them at the same time. In the beginning, this takes a bit of getting used to, as I'm doing here managing multiple instances of Unity at the same time, ensuring that this all checks out. But ultimately, this is a really great workflow on structuring your packages in this way and on just referencing them into multiple Unity projects.

Usually, we have a little bit different structure for different packages, but usually this is also reflected in source control. For example, this is our main entry point for the maintenance of these packages. All the actual packages are submodules here, and the projects are managed in this wrapper here. Often, some of these packages in themselves have their own test projects. For example, Shader Graph Markdown is a package that's on GitHub and open source, but there's also a test project for it that has again this same structure of various individual Unity projects with different feature sets. For something like Shader Graph Markdown, it's important that it works in URP, it works in Built-in, it needs to work in projects that have both of these, and so on. There are even more permutations to keep track of. It would be impossible or very hard to always change this in one project. Say you only have one project and you want to test all these changes, you're continuously removing HDRP, adding URP, and all these kind of things. This becomes old very fast. Instead, this compositional approach just allows you to say: here's the source code of this package, and I'm using that code across all these various Unity projects.

Here was an overview of that. The way this works on a code level is you have your #if defs, you have assembly definition files. I'm not going to go into those here, but basically you can make your code work for various breaking changes in other packages in Unity core by being careful about what you're adding there in terms of keywords.

Thank you. This is one part of project composition. The other part is how you're actually working with this in productions, like when you're working on a game. What we found has actually happened for a lot of the projects that we're working on is the Assets folder, the good old Assets folder that everything has happened in over the years, is becoming less and less important. Everything happens in packages because this composition approach is so strong for workflows. For example, even in projects that have art assets and code assets, we often separate those into separate packages and just say: okay, there's one package that has the art assets and one package that has the code for a specific project. Even if those are not on a registry, they're just in your regular repository, they're just in different folders, same as we did earlier with that my package, our simple one. This allows you, for example, to have thin Unity projects that wrap around these packages. What you can do, for example, is you have your full project where you say this is where art and code come together, but you also have a continuous integration project that doesn't have the art assets, it only has the code packages. Obviously, the project itself will not work if you don't have all those art assets, but you can very well run all your unit tests or you can run specific smaller scoped scenes and all that kind of stuff, and you're not getting all the asset import times. On some projects, we've seen this as far stretched as artists are working in one Unity project and developers are mostly working in another one. But since they're sharing packages and sharing prefabs and all that kind of stuff, everything keeps working. You're not getting heavy import times because the artists have added three gigabytes of textures, and the artists are not getting immediately compiled errors if the developers have messed something up. You separate this more by moving it into these separate packages.

This is also cool because when you're running into problems with a Unity subsystem or a specific platform or whatever, what we often end up doing is we're spinning up a new project, just adding the packages that we suspect are responsible for producing that issue, and testing with that. That cuts down iteration times, it makes it easier to actually find a problem, pinpoint where it comes from. That's the other side: two parts of composition here. One is composition of these kind of test projects where you say you want to have your packages in various versions of Unity, and the other is composition in production projects where you usually have the same Unity version for everything, but you're combining packages differently depending on what these thin projects do. The Assets folder in that case mostly becomes a place for project-specific configuration files, like XR setup or specific Scriptable Objects that control how things behave, or project IDs and all that kind of stuff. These stay in the Assets folder, but by now for even super large projects, we basically have empty Assets folders without code, without assets, without anything but these configurations.

From here on, I have to choose in the few minutes I have left. With this many topics, I could go into one thing I want to quickly show, I think, is something about dependencies. If you're working with packages, then ultimately you're going to get at a point where you have dependencies between these packages. Usually that's called a root package or a leaf package, depending on does this package have other packages that depend on it. This is always an interesting design choice, especially if you're getting started bringing your company's code to a package registry, on how much or how long do the chains of dependencies get for stuff, and how much do you try to have kind of a shallow hierarchy of things. It's an ongoing challenge also for Unity's own package system. There was a point in time when they were relatively not careful enough with that, and there were packages springing up left and right. In some areas, they have a dependency problem basically, where there are so many dependencies that it gets hard to do any meaningful changes. You would have to do changes across the board to do that. This is one of the reasons why they, for example, have started moving the graphics code back into the core of Unity and not as packages on a registry anymore, because they're always changing those in lockstep and that gives them less dependency headache.

What we tried as an experiment quite a while ago is visualizing those dependency structures. I can share an interesting example that maybe gives some insight into development delays of some of the systems that Unity has been working on in recent years. This is a graph of dependencies. It's basically a tool that we've been building just for seeing how packages are connected to each other. What you can see here, it's all tiny, but for example this is Entities. We're in the graph for 2019.4 now, where basically Unity was still saying okay, Entities is right around the corner, it's going to release any minute now. There's an interesting visual difference between the upper part of this graph here, which looks super messy, like there's so many connections going on, and the lower part of the graph here. The difference is that the upper part is everything related to Entities and everything related to Burst and Jobs and these kind of things, with all the various things that have appeared about DOTS runtime, about Hybrid Rendering, and all that kind of stuff. It's super messy. This was already obvious from just looking at the dependencies that this is nowhere ready for production. Basically, whenever you see multiple versions of something in these stacks here, it means that across these packages, there's no understanding of what the actual shared dependency is. When you're pulling in multiple projects, you suddenly end up with packages that were tested against Jobs 2.4 and packages that were tested against Jobs 2.7, and so on. All of these have different dependencies. Someone didn't quite get a hold of keeping all these threads together and keeping the dependencies in check. In the old Unity world, that all looks more orderly. You have pretty short chains, you have not as many overbookings as what we call them, on how many versions of this does it need to be tested against at the same time. Everything looks a lot more orderly.

If you compare that with 2021.2, I think this is just a random 2021.2, here they made the hard decision to postpone Entities officially and remove it from Package Manager entirely to give them more time on fleshing them out. I think now in the end of 22 cycle, 2022 cycle or 2023 cycle, we're seeing the first fruits of this on a more ordered version of these packages coming back. Lots of stuff has moved into the core, someone probably went back to the drawing board about dependencies, and you get more orderly hierarchies on how things depend on each other. While this is mostly a curious example, I think it's always interesting to look at packages if you are structuring them for an organization on these kind of dependency chains. How do you want this to look in the end? Do you want long chains or do you want shallow chains? Do you want to make sure your packages support a wide range of other dependencies, or do you want to make sure they mostly work with individual ones? There's no one-size-fits-all rule to this. It's something that you have to figure out for your specific organization, and it depends on things like how many simultaneous Unity versions do you need to support.

With this, I think we went from super simple examples on how packages work and how they look in Package Manager, all the way to complex dependency chains and looking at how messed up this entire Entity system was in 2019. Quite a ride, as promised. Feel free to reach out to me or to Needle in general if you were planning to work with packages, if you want to use some of our tools more. A lot of it is open source, other parts are not. I would have had more things here, but we're out of time, so I'll have to save that for a later talk.

I can make that available right now. It's relatively old tooling, it was an old experiment basically, and I never got around to just releasing it. But I can do that. There's nothing secret in there, it just parses the package dependencies and shows them with a graph view. So yeah, I'll make sure to upload that somewhere and tweet about it.

One reason there is we're doing that, for example, whenever we're downloading something from the Asset Store. The Asset Store is kind of this old world by now where you are cluttering your project. We all know that you download something from the Asset Store and it's kind of a gift box of where will this tool unpack in my project and what kind of havoc will it wreck. I look very much forward to a future where everything is neatly structured in packages that don't mess with each other. The hybrid packages approach is kind of a stepping stone in that direction. Originally, we really built it to be able to release UPM packages on the Asset Store, because Unity has been saying that they want to get this production ready for I don't know five years, since packages exist now. We got impatient and found this interesting workaround. The cool thing is that they adopted it and now make that... there was quite a bit of discussion on, hey, you really found a problem, or you could consider it the fact that Unity packages allow you doing that unpacking into different folders than they were intended for. But it turns out to be super powerful for structuring in this way. This is really the gist of it. It was originally intended for shipping things on the Asset Store while packages are not fully ready there. But we've seen that companies have adopted this as well because it makes it relatively easy, if you don't have a registry for example, to ship packages to people by just giving them a file. You give them a Unity package and they can unpack that and they have a new version of something. It's easier than a lot of the alternatives. You don't need to send someone a zip file anymore, and it's still cleanly put in one place, it has a clean version, you can be sure that you are not introducing unwanted files and moving things all around. Lots of structural improvements there. But if you have the choice, I would always recommend put things on your own registry, make them immutable, and maintain them properly as UPM packages.

Since this is based on, we're not reinventing the wheel or anything, this is what npm is doing for authentication against registries. All these packages in Unity do is add a nice interface so that you're able to log into it and use credentials. This works with JWT if the registry is set up to do authentication with that. Other registries are set up to just do simple bearer token authentication. So it really depends on how you're setting this up. Depending on how it works on your registry, you can also do it differently. For example, at the Needle registry, we allow you logging in with your GitHub account, and then you get your token. In the configuration of the registry, you can say okay, which GitHub organizations have access to which packages or which accounts have access. So you can fine-grained manage who can do what.

This is one of the slides I actually didn't get to here. There's many I didn't get to, but one is automation. Basically, at Needle, we're always trying to add automation as another layer on top of something that you could also do manually. I find this is super useful to be able to test things locally and not just like, dear server, here's my new version, please tell me if there's any errors. You want to be able to replicate things locally. What I showed you basically is the setup and structure and how you can do that manually, which is great if you're actually developing a package. You're doing changes on something that needs to work across seven different Unity versions and URP versions, and the speed of this is just the fastest you can get. You're tapping into another Unity instance and you see if that code you just wrote works in 2019, 2020, 2021. Of course, if your package is in a more stable shape and you're doing mostly bug fixes, this is a good time to automate this. While not public, we have a couple of in-house tools to basically run Unity's unit tests across these projects. The idea is you have this structure, you have your seven different Unity projects, and we're just running the tests in them automatically as you're making commits.

We're mostly using Unity's infrastructure for unit tests. There's actually two: there's the integration for the Test Runner, and there's an entire suite for graphics tests, like there's a Graphics Test Framework that you can use as well if your packages are more graphics heavy. Basically, what we're doing in our automation is in your projects, all the tests that you have set up are run as part of automation in Unity's regular system. Again, it means you can do it manually. Something fails, that's no problem. You open that Unity project on your local machine, you run the tests manually, and everything is just a click away on reproducing what you saw as a test result. I like that much more than what I've seen, especially some large companies have been adopting an approach of kind of fire and forget. You're doing your change, you're pushing that somewhere, and then two hours later you get the results on, oh okay, now everything is red. I think that's not a great approach for stable development of stuff. We're more in this camp of you should be able to replicate it locally, but you can add automation on top.

What do you mean with adopting these tools? They are all in active production, right? I guess that if you're seeing all these packages, quite often people consume these packages without really looking into them. You're obviously a bit more hands-on with the packages there. So do you know if some of the dependencies are picking up more dependencies, and the visual scripting, Bolt, Shader Graph, and VFX Graph are starting to get more used in some of these packages?

I don't know much about the visual scripting side. I have used it a couple of times, but then not for a while because it wasn't in a great shape then, and I haven't picked it up again. But for Shader Graph and VFX Graph, it's always interesting to look at what Mirror logs in terms of what changes have been done, what progresses forward. I think what's always interesting to see is if there are unexpected bug fixes in patch releases, like URP 10.11 or something like this, and then you suddenly have a couple thousand lines of changes on fixing a particular problem. These kinds of insights are super helpful because they often uncover problems that have been there. They give more reasoning on should I update my project or not, which is something that I find particularly hard with the opaqueness of how Unity approaches packages on this: here's a new version, we're not really telling you why that is there. So yes, I think there's lots of momentum around Shader Graph and the SRPs in general on getting that more streamlined. One unfortunate thing I think is that some of the SRP development has moved back to being core packages, so that means they're not on the registry anymore, they're shipping with the core. We can still track changes between them, but it's not as easy anymore to update in the same Unity project between different URP versions, for example. We actually fought quite a bit for getting versions back into these packages that now ship with core. There was a time when Unity started shipping everything as 12.0.0, next version 12.0.0, and so on. They said that makes sense because it's shipping in core, but it's impossible if you're actually doing modifications to see that something has changed.

Foreign music.