Optimizing memory usage in LUA

Couple of days ago I received a message on curse.com about the memory usage of UberInventory. The general statement was that UberInventory uses a lot of memory. We were aware of this issue, but we had never given it much attention. But after this message we could ignore it no longer.

We have been reading several articles on the internet on how to optimize memory usage, performance and garbage collection using LUA. All those documents basically state that using locals as much as possible improves performance when accessing data structures and when storing a lot of data most of the times it is better to create larger tables instead a lot of smaller tables. This some times leads to a little more unreadable data structures but it keeps garbage generation to a minimum and less memory is needed to store the data.

Since UberInventory contains a lot of static data (pricing and recipe data) our first focus was to see if and how we could optmize those data structures.

The following results have been generated using LUA 5.0.1 and we have used the gcinfo() function to produce the numbers.

Orignal version of UberInventory_prices.lua

	Dynimac Memory	Garbage threshold
Before	1531	3062
After	4189	6124
Delta	2658	3062

UBI_Prices = {
 [25] = {nil,7},
 [35] = {nil,9},
 [36] = {nil,7},
};

Modified version of UberInventory_prices.lua

	Dynimac Memory	Garbage threshold
Before	1060	2121
After	2500	2121
Delta	1440	0

UBI_Prices_Buy = {
 [38] = 1,
 [39] = 5,
 [40] = 5,
};
UBI_Prices_Sell = {
 [25] = 7,
 [35] = 9,
 [36] = 7,
};

As you can see with the modified version there is big drop in the total memory that is needed to store the data and the structure does not create any additional garbage.

Changes made to UberInventory_prices.lua

• Split data into two tables, one for buy prices and one for sell prices

Original version of UberInventory_recipes.lua

	Dynimac Memory	Garbage threshold
Before	961	1921
After	3160	3842
Delta	2199	1921

UBI_Creatures = {
 [66] = { name='Tharynn Bouden', area='Elwynn Forest' },
 [340] = { name='Kendor Kabonka', area='Stormwind City' },
 [450] = { name='Defias Renegade Mage', area='Westfall' },
};
UBI_Quests = {
 [22] = { name='Goretusk Liver Pie', area='Westfall' },
 [38] = { name='Westfall Stew', area='Westfall' },
 [90] = { name='Seasoned Wolf Kabobs', area='Duskwood' },
};
UBI_RecipeDrops = {
 [4213] = { {8716,1}, {12396,1} },
 [4408] = { {7800,1} },
};

Modified version of UberInventory_recipes.lua

	Dynimac Memory	Garbage threshold
Before	636	1272
After	1230	1272
Delta	594	0

UBI_Creatures = {
 [66] = 'Tharynn Bouden|Elwynn Forest',
 [340] = 'Kendor Kabonka|Stormwind City',
 [450] = 'Defias Renegade Mage|Westfall',
 [590] = 'Defias Looter|Westfall',
};
UBI_Quests = {
 [22] = 'Goretusk Liver Pie|Westfall',
 [38] = 'Westfall Stew|Westfall',
 [90] = 'Seasoned Wolf Kabobs|Duskwood',
};
UBI_RecipeDrops = {
 [4213] = { '8716:1', '12396:1' },
 [4408] = { '7800:1' },
};

Here you see the same results, huge drop and no garbage. These changes combined resulted into a decrease in memory of between 2 and 3 MB just for the static data.

Changes made to UberInventory_recipes.lua

• For creatures and quests combine the name and area into one text field instead of using a table structure
• For drop info combine mob id and drop rate into one text field instead of using a table structure

All the changes described in this post are implemented for UberInventory 1.6.

The next step will be digging into the data structures of the dynamic data stored within UberInventory.lua (WTF\Account\…\SavedVariables) and optimize those for memory usage and garbage.

UberInventory 1.6

We are currently working on UberInventory 1.6. At the time of this writing we have not yet set any release date.

Following is a list of tasks we are working on;

• Fix issue that occurs on fresh installs (or joining a new guild) of UberInventory when visiting the guild bank

• Redesigning the settings module
• Adding more options to customize the behavior of UberInventory
• Improving overall performance (specifically determine usability of items)
• Removing obsolete code

Upcoming version of UberInventory

Over the time we have received requests from users that they would really like to be able to search across multiple characters, guildbanks or all the data that has been collected.

Within version 1.3/1.4 of UberInventory we started to make some of the necessary changes to the way we build up the user interface filters and how they are linked back to the data structures used for storing information on items. Over the last couple of days we have been working on making the modifications necessary to also make it happen from a coding point of view.

The new location filter will look as follows

All items – Combination of all characters and all guildbanks, including the current character

Maymayhem – My current logged in character, will display items from all locations
  Bag – Show items stored in the bags of the current logged in character
  Bank – Show banked items of the current logged in character
  Keyring – Show items stored in the keyring of the current logged in character
  Mailbox – Show items stored in the mailbox of the current logged in character
  Equipped – Show equipped items of the current logged in character

All Guildbanks – All guildbanks combined
  The Mystics – Items stored in The Mystics guild bank
  Wolf Pack – Items stored in the Wolf Pack guild bank

All Characters – All characters combined, including the current logged in character
  Drexore – Items from the Drexore character
  Galure – Items from the Galure character
  etc.

This preview demonstrates the new functionality within the UberInventory interface

At the moment there are still some issues that need to be tackled;
- Show correct cash totals when selecting combi items (All items, All Guildbanks and All Characters)
- Display proper info within tooltips
- Do some in-game testing
- Warn if there are 50 messages stored in the mailbox. WoW only show 50 mails even if you have more mails waiting.

We are making good progress and are hoping to be able to provide you all with a new version of UberInventory before Christmas.

The new version will also contain an updated version of the static data for prices and recipe data, and now contains information on book related items (like Gift of the Wild).

Fixing UberInventory

Spent a lot of time trying out several things to resolve the earlier mentioned issue I encountered when the InfoPanel Addon was installed.

The image on the left show to incorrect tooltip, the one on the right displays the correct version of the tooltip.

I finally ’solved’ this issue by adding the following code after the real code that adds all the wanted information to the tooltip and just before calling GameTooltip:Show()

local line = getglobal( GameTooltip:GetName().."TextRight"..GameTooltip:NumLines() );
line:SetWidth( line:GetStringWidth()+16 );
line:SetHeight( 12 );
line:SetJustifyH( "RIGHT" );

I haven’t released a new version of UberInventory yet on Curse.com, first need to do a bit more testing to see if this really resolves the issue. But so far it appears to be working

Updated to BagTips and MailTips

Yesterday or better said early this morning I have updated two of my World of Warcraft Addons, BagTips and MailTips.

I had installed the Addon InfoPanel and from that moment one the tooltips displayed by both BagTips and MailTips did not like right anymore. The names of the items from either a bag or a mail message was displayed using two lines within the tooltip instead of just one. After a bit of searching I found that InfoPanel uses the pipeline character in some displayed text (plugins RaidInfo and Zone).

InfoPanel_SetText(f,
                  "%.1f |cffffff00|||r %.1f",
                  x * 100,
                  y * 100);

Somehow this has an influence of information added to tooltips and fontstrings that use the texture tag to add images to textstring

tooltip:AddDoubleLine( "  |T"..value.texture..":0|t "..key,
                       value.count,
                       r, g, b );

It appears that when you are using the method and using the code InfoPanel uses the actual width for the fontstring is not calculated correctly. To quickly workaround this I just add two spaces after the item name like so

tooltip:AddDoubleLine( "  |T"..value.texture..":0|t "..key.."  ",
                       value.count,
                       r, g, b );

and everything works again as expected.

Now I only need to fix UberInventory, because it has similar issues. Hopefully I can get those issues resolved today as well.