[Technical] Optimization for AO

Game Discussion
1

The contents of this post still apply after the Empires Update. Please check for any later replies I make under this thread for information not important enough to update this topic.

Decided to make a topic about my findings on performance/optimization on AO today. A lot of the stuff here might just straight up be wrong so don’t take my word for it!

Note before going ahead: all of the information I’ve collected here come from 1 MicroProfiler dump and should not be used as a single source of evidence for any problems with AO’s optimization. It is important to test on multiple times and on multiple devices.

What I found

When I dumped the MicroProfiler and opened up the memory flamegraph, I noticed that the thing taking up the most memory allocation was “new Texture”. I’m not too aware if the “new” prefix refers to the construction of new instances, but I am going to assume such :sob: , so does the game have a bunch of texture instances, and if so wouldn’t it be possible to use custom materials which got introduced instead? I checked Devas of Creation, another big MMO-RPG on Roblox, as well as the game I program for, PSYCHIS, and I wasn’t able to find any “new Texture” timer/thread - all of this could definitely be a whole bunch of non-sense but i want to mention it anyways.

(Memory flamegraph by memory allocations size for an AO client session ran without preforming any actions after ~2-3 minutes)

image
image1057×224 6.47 KB

Another thing related to memory which i have no proof/information for, but I’ve noticed depending on the server the player is in, memory can vary from about 2700-3000MB - at least during the first few minutes of just having had loaded in - which leads me to believe that there is definitely something server-related (possibly with instances not being destroyed/cleaned up?) causing this memory consumption. Vetex is probably already aware of this.
I honestly don’t have anything more to say about memory related stuff, I can definitely look into it more, but I probably wouldn’t be able to find much more at least memory related. Here’s another memory flamegraph if it finds any use though - these tests have already probably been conducted anyways.

(Memory flamegraph by memory allocations count for an AO client session ran without preforming any actions after ~2-3 minutes)

image
image1023×246 12 KB

As for analyzing the CPU flamegraph, many of the threads present would be fixed if there weren’t as many humanoid instances in the game - as far as that is what I have come to believe. If it’s possible for them to de-load based on distance and sight, it would do great work. I believe Vetex has already made this optimization as said on the trello that “player characters now unload based on distance from your own”, but I am unsure if this includes NPCs as well if they haven’t already been doing this. If they are, it would be definitely good to check as to why humanoids are being such a detriment to client performance despite being de-loaded.

(CPU flamegraph for an AO client session ran without preforming any actions after ~2-3 minutes)

image
image1123×277 17.2 KB

Anyways, as for networking, I wanted to see if AO uses any networking library. I obviously cannot see through the MicroProfiler, so I am wondering if AO does. If it doesn’t, adding one definitely could be good. One that I use live-in-production for the game I work on, is ByteNet by ffrostfall. It’s great.

image
image714×489 67.9 KB

It can actually help client performance - probably not in AO’s case since the problems stem from other places - but more importantly it helps with ping.

Final remarks

Anyways, that’s all I’ve got for now. I’ll update this topic/thread with any more of my findings/suggestions/questions.
If anyone has found anything else, add them here. Or if anyone has any questions or things you want me to look at - as long as they’re within my reach - I am more than happy looking into them!

Resources

Some articles definitely worth taking a look at, and I’ll be adding to these as I find more WELL WRITTEN ones.

Device Specifications

These are my specs I used to conduct these tests:

CPU: AMD Ryzen 5 5600X 6-Core @ 3.70 GHz
GPU: Nvidia RTX 3070
RAM: 32.0 GB
37 Likes
2

Another suggestion out of suggestions :pensive:
/j

3

Don’t have suggestions permissions :pensive: Hopefully someone can move it to suggestions.

4

Nah, they ain’t that kind. The only thing that might save this topic from immediate death is that it’s about programming optimization, and most people here will be too intimidated by all these complicated graphs and stuff to question it.

2 Likes
5

I really doubt they would consider this a suggestion

3 Likes
6

to ramdoys:

I FINALLY FOUND YOU!!!

1 Like
7

i feel like i should ping vetex to read this but i dont wanna get banned

8

This topic is definitely not at a “good” state yet - I bet a bunch of this stuff could probably be disregarded once the Empires Update releases. If you do ever wanna do something like that, do it once its matured a little bit :sob:

9

maybe analyze data usage 2 weeks or so after empires releases? that’ll give time for any hotfixes vetex plans on

1 Like
10

Empires is in like, 2 days, so after that you can test again and compare to see if it has improved at all. Also, there isn’t enough time for this anyways, and since Full Release comes next. It’s best to look at this with up to date data.

1 Like
11

I’ll definitely try doing that. Really there isn’t that much I can add to this thread except recommend things that Vetex could already be doing and things that are obviously bad for performance appearing on the MicroProfiler. Many of the great tools Roblox provides like extending usage of the features the Developer Console has (e.g. LuauHeap) or the analytics provided by Roblox through the Creator Dashboard. If Vetex gave me the opprotunity to help out with optimization, I’d take it in a heartbeat. There’s only so much I can do here lol.

5 Likes
12

TBF it would be fitting of western devs to never optimize

2 Likes
13

western game dev was here

1 Like
14

You know what i think we should give you user of the month for this

2 Likes
15

Genuine question that no one has asked yet. I don’t have much experience dealing with networks and all of that. However, i assume the main reason AO is sending so much more data is because it might not be using ByteNet with uint8. Although, unsigned integers at what I assume to be 8 bit, wouldn’t this slow down the loading for things in-game? Assuming AO uses a larger unsigned integer for loading stuff in.

Another thing too that I’d need clarification on (again, I don’t know too much about network stuff), how will this handle things like larger numbers? Those are more than 8 bits in size, unsigned 8 bits can only reach 255, and everything is probably 32 or 64 bit. Would this just split up damage into 4-8 instances, or I guess it’d all be in one packet or something.

It could also be that the solution is quite literally just packets which are being done anyways, and the way it’s being done right now it’s probably sending data in larger sizes. eg. 12 bits are fine, so send 2 8 bits in a packet with instructions to combine them later; current would be sending a 32 bit instruction and be wasting a whole 16 bits that could be used elsewhere. This is obviously just an example.

16

It says using an uint8 because the packet was created and defined as a uint8 datatype. You can change the datatype to be almost all of the ones available on Roblox. So I could do ByteNet.inst or ByteNet.int32, or Bytenet.string. Its a library made by someone, not Roblox, and many more exist out there.

22

first time seeing someone actually criticize performance correctly :face_holding_back_tears:

6 Likes
23

this is one of the few posts where pinging vetex so he sees it would be appropriate

mandatory /hj so I don’t get taken out back and shot

24

Wanted to add on to this and say that ByteNet uses buffers on defined types to minimize the amount of data being sent client ↔ server. Even if AO had a custom networking solution, I don’t think it uses buffers as buffers were only added to Luau some months ago I believe. Just remember networking stuff like this is honestly overkill, and only really needed when ping is that important - such as in a game like AO.

25

Did Vetex ever work on optimization? I genuinely forgot