Modmenu pagination by Aurumbi · Pull Request #123 · AWSW-Modding/AWSW-Modtools

Aurumbi · 2026-02-17T13:43:40Z

Added pagination to the in-game mod browser, along with fixing certain issues regarding to getting modlists from steam.

Main Changes

Fixed `steamhandler`'s `QueryApi` method

This method, which is used to get sections of the modlist from steam's API, has a tendency to crash silently on repeated calls, which also crashes the game. The source of this crash seems to be related to steam caching these results, as these crashes persist between play sessions, and occur consistently from the second call onwards for about 15 minutes after a successful call.
This was solved by created an on-disk cache of this method's results, and using that cache until that 15-minute interval elapses, which allows the QueryApi calls to never use the problematic steam cache.

Added pagination to the In-game mod browser

With the increasing number of mods available in the game since the development on the browser, both load time and navigation go slower as a consequence of the longer modlist. pagination was then added to both allow partial loading of the modlist, and to allow quicker navigation of the modlist.

note: While partial loading is now possible, It has not been implemented. with the loading improvements implemented in the next section, it has been deemed not necessary.

Load speed improvements

The critical path of getting the modlist (which defines the content of the browser) has been examined, and certain optimizations have been made. of those there are:

The modlist is preloaded and cached on startup, which saves significant time on the first opening of such list.
Certain debugging information has been reduced to it's identifiers, significantly reducing IO.

muddyfish · 2026-02-17T14:24:48Z

modloader/__init__.py

-        steammgr.CachePersonas()
+    # if has_steam():
+    #     steammgr = get_instance()
+    #     steammgr.CachePersonas() #TODO: Change to call the new thing...


Either fulfill the TODO, or remove the comments.

Suggested change

# steammgr.CachePersonas() #TODO: Change to call the new thing...

# steammgr.CachePersonas()

muddyfish · 2026-02-17T14:25:28Z

modloader/modconfig.py

 if workshop_enabled:
    from steam_workshop.steam_config import has_valid_signature
    import steam_workshop.steamhandler as steamhandler
+    import steamhandler_ex


Can we choose a better/more readable name for this?

yeah, absolutely.
I'd propose either cached_steamhandler or steamhandler_extensions, depending on whether we plan to extend it more in the future...
I'm leaning towards cached_steamhandler, simply because I hope we won't need to wrap/modify more on this low level.

muddyfish · 2026-02-17T14:25:53Z

modloader/modconfig.py

            mods[-1][3] += "\n\nVerified by {}".format(verified.username.replace("<postmaster@example.com>", ""))
        else:
-            print "NOT VALID SIG", mod
+            print "NOT VALID SIG", mod[1]  # Note: printing only the mod name, instead of the whi=ole thing SIGNIFICANTLY speeds up this call


This is slightly worrying - how many mods don't have a valid signature? I thought I'd signed most of them

These 15 fail, though I haven't checked why

(3336106514) ALL WORKING MODS! 32 (Flight of Love doesn't show)
(3651582794) Angels With Scaly Wings: Chinese Simplified
(3666234554) Card Game Expanded 卡牌游戏增强（Two Languages Support）
(2801825863) Casual Arson
(2697982171) Casual Vandalism
(2096774388) Lorem_RPG
(2742645268) MagmaClient
(2990207385) Meet Naomi
(1305731599) Modtools
(2766323849) Name Re-entry
(3465212790) Naomi Maze Skip
(2736325690) Skip Credits
(2987066957) The Morning After
(3645564443) The Traitor
(3657449256) Vykoupení

muddyfish · 2026-02-17T14:27:04Z

modloader/steamhandler_ex.py

+                # else: already done
+
+                # Write cache file
+                with open(cache_file_name, "wb") as cache_file:


Can we use literally anything except pickle? It's very insecure.

Yeah. I used pickle because it was the simplest way to store and restore values so that they behave the same, but I have no attachment to it. I'm afraid I don't quite get what the security issue here is, so what would you suggest to use here? would a tuple of dicts be better (with adapted __getattr__ so that they behave like the ctype results)?

muddyfish · 2026-02-17T14:27:38Z

modloader/steamhandler_ex.py

+        if not isinstance(steam_manager, SteamMgr):
+            raise TypeError("steam_manager must be a steam_workshop.steamhandler.SteamMgr instance!")
+        self._steam_manager = steam_manager
+        self.threads = [] # This can create threads in QueryApi, which will persist after the call returns (as is required).


This list only grows, never shrinks. Is this AI written?

This was mostly not thought out well... Because of how QueryApi (and all of steam DLL functions) work, if the cache file is used then we need to create a thread to execute the query callbacks in order to imitate the behavior of steamhandler.QueryApi. These threads also need to live beyond the end of the function. I was concerned that these threads could be difficult to track (especially if some of the callbacks start misbehaving), so I thought "why not just chuck them into a list, so that we at least know which ones have been created?". but then I didn't really have much to do with them, so the list became kinda pointless.
Additionally, I avoid using AI for my work, so any stupid decisions are my own.

muddyfish · 2026-02-17T14:37:59Z

modloader/steamhandler_ex.py

+        print "Called cached queryAPI with page={}".format(page)
+
+        # Get most recent cache file if exists
+        cache_file_name = self.get_cache_filename(page)


Why are we storing a cache in a file rather than say in memory?
Given that it expires in 15 minutes, that's not particularly useful for multiple play sessions

The issue I was running into (which was the main one preventing the game from starting) Was that repeated calls to QueryApi failed silently, and in doing so, crashed the game. Once these start to fail, they keep failing even between play sessions until about 15 minutes pass, which prevents anything from accessing the modlist until that time passes. Therefore, I use a file to store those results, as they need to be accessible between play sessions.
I've added a description of this in the PR's description, as this is the most significant change.

muddyfish · 2026-02-17T14:38:41Z

modloader/steamhandler_ex.py

+                    cPickle.dump(tuple(islice(array, arr_len)), cache_file)
+
+            except Exception as e:
+                print "Cache callback raised Exception:", str(e)


Why just printing here?

I honestly don't remember, I think it's a holdover from testing? We can reraise, or just remove the except case altogether.

muddyfish · 2026-02-17T14:39:47Z

mods/core/download_mods.rpy

+
+    from modloader import modconfig
+
+    # Cache mod validity to expedite mod browser startup


I don't understand this.

The call to modconfig.steam_downloadable_mods still takes a few seconds on the first call (as after that it's cached in memory), so I chucked it on a thread during startup so that it would already be done before the user ever wants to access the mod browser.
A more robust way of doing this would probably be to make a load_steam_downloadable_mods function which loads this data on a thread, then setup steam_downloadable_mods so that it waits until that function finishes. I honestly should have thought of that sooner...

muddyfish · 2026-02-17T14:40:04Z

mods/core/download_mods.rpy

+        renpy.show_screen('modmenu_paged_modlist', contents=modlist[start:end], use_steam=use_steam)
+        return
+
+# Previous implementation kept, as the scrolling may be useful in the future


Git allows us to go backwards if we need to.

Fair. you can blame that on me not being good with git...

Aurumbi · 2026-02-17T17:38:01Z

mods/core/download_mods.rpy

+# Previous implementation kept, as the scrolling may be useful in the future
+# screen modmenu_download(contents, use_steam):
+#     modal True
+#
+#     frame id "modmenu_download" at alpha_dissolve:
+#         add "image/ui/ingame_menu_bg3.png"
+#
+#         add "image/ui/ingame_menu_bg_light.png" at ingame_menu_light
+#
+#         #Title
+#         text "MOD MENU":
+#             size 65
+#             xpos 0.5
+#             ypos 0.05
+#             xcenter 0.5
+#             yanchor 0.5
+#             font "Ardnas.otf"
+#
+#         #Close Button
+#         imagebutton:
+#             idle "image/ui/close_idle.png"
+#             hover "image/ui/close_hover.png"
+#             action [Show("modmenu", transition=dissolve),
+#                     Hide("modmenu_mod_content", transition=dissolve),
+#                     Hide("modmenu_download", transition=dissolve),
+#                     Stop("modmenu_music", fadeout=1.0),
+#                     Play("music", "mx/menu.ogg", fadein=1.0),
+#                     Play("audio", "se/sounds/close.ogg")]
+#
+#             xpos 0.94
+#             ypos 0.02
+#
+#         frame:
+#             background None
+#             yminimum 900
+#             ymaximum 900
+#             xmaximum 425
+#             xminimum 425
+#             xpos 65
+#             ypos 90
+#
+#             #button hieght 125
+#             vpgrid id "modselect_vp":
+#
+#                 yminimum 900
+#                 ymaximum 900
+#                 xmaximum 425
+#                 xminimum 425
+#
+#                 cols 1
+#                 spacing 30
+#                 draggable True
+#                 mousewheel True
+#
+#                 for modid, name, author, description, url in contents:
+#                     $ modname = modmenu_name_cleaner(name)
+#
+#                     if len(modname) <= 21:
+#                         #if mod is installed
+#                         if str(modid) in modinfo.get_mod_folders():
+#                             $ modname = modname + "\n{size=-5}(Installed){/size}"
+#                         #if mod is not installed
+#                         else:
+#                             $ modname = modname
+#
+#                     #if modname is greater than 21 characters, decrese size of font by 5
+#                     elif len(modname) <= 25:
+#                         #if mod is installed
+#                         if str(modid) in modinfo.get_mod_folders():
+#                             $ modname = "{size=-5}" + modname + "{/size}" + "\n{size=-5}(Installed){/size}"
+#                         #if mod is not installed
+#                         else:
+#                             $ modname = "{size=-5}" + modname + "{/size}"
+#
+#                     #if modname is greater than 25 characters, decrese size of font by 10
+#                     else:
+#                         #if modname is greater than 30 characters, decrese size of font by 10 and cut all text after 30 places
+#                         if len(modname) > 30:
+#                             $ modname = modname[:30]
+#
+#                         #if mod is installed
+#                         if str(modid) in modinfo.get_mod_folders():
+#                             $ modname = "{size=-10}" + modname + "{/size}" + "\n{size=-5}(Installed){/size}"
+#                         #if mod is not installed
+#                         else:
+#                             $ modname = "{size=-10}" + modname + "{/size}"
+#
+#
+#                     textbutton "[modname]":
+#                         style "modmenu_select_btn"
+#
+#                         action [Hide("modmenu_mod_content"),
+#                                 Show("modmenu_mod_content",
+#                                      modid=modid,
+#                                      name=unicode(name, "utf8"),
+#                                      author=unicode(author, "utf8"),
+#                                      description=unicode(description, "utf8"),
+#                                      url=url,
+#                                      use_steam=use_steam,
+#                                      transition=dissolve),
+#                                 Play("audio", "se/sounds/open.ogg")]
+#
+#
+#         bar value YScrollValue("modselect_vp"):
+#             style "modmenu_select_slider"
+#             #yalign 0.95


Suggested change

# Previous implementation kept, as the scrolling may be useful in the future

# screen modmenu_download(contents, use_steam):

# modal True

#

# frame id "modmenu_download" at alpha_dissolve:

# add "image/ui/ingame_menu_bg3.png"

#

# add "image/ui/ingame_menu_bg_light.png" at ingame_menu_light

#

# #Title

# text "MOD MENU":

# size 65

# xpos 0.5

# ypos 0.05

# xcenter 0.5

# yanchor 0.5

# font "Ardnas.otf"

#

# #Close Button

# imagebutton:

# idle "image/ui/close_idle.png"

# hover "image/ui/close_hover.png"

# action [Show("modmenu", transition=dissolve),

# Hide("modmenu_mod_content", transition=dissolve),

# Hide("modmenu_download", transition=dissolve),

# Stop("modmenu_music", fadeout=1.0),

# Play("music", "mx/menu.ogg", fadein=1.0),

# Play("audio", "se/sounds/close.ogg")]

#

# xpos 0.94

# ypos 0.02

#

# frame:

# background None

# yminimum 900

# ymaximum 900

# xmaximum 425

# xminimum 425

# xpos 65

# ypos 90

#

# #button hieght 125

# vpgrid id "modselect_vp":

#

# yminimum 900

# ymaximum 900

# xmaximum 425

# xminimum 425

#

# cols 1

# spacing 30

# draggable True

# mousewheel True

#

# for modid, name, author, description, url in contents:

# $ modname = modmenu_name_cleaner(name)

#

# if len(modname) <= 21:

# #if mod is installed

# if str(modid) in modinfo.get_mod_folders():

# $ modname = modname + "\n{size=-5}(Installed){/size}"

# #if mod is not installed

# else:

# $ modname = modname

#

# #if modname is greater than 21 characters, decrese size of font by 5

# elif len(modname) <= 25:

# #if mod is installed

# if str(modid) in modinfo.get_mod_folders():

# $ modname = "{size=-5}" + modname + "{/size}" + "\n{size=-5}(Installed){/size}"

# #if mod is not installed

# else:

# $ modname = "{size=-5}" + modname + "{/size}"

#

# #if modname is greater than 25 characters, decrese size of font by 10

# else:

# #if modname is greater than 30 characters, decrese size of font by 10 and cut all text after 30 places

# if len(modname) > 30:

# $ modname = modname[:30]

#

# #if mod is installed

# if str(modid) in modinfo.get_mod_folders():

# $ modname = "{size=-10}" + modname + "{/size}" + "\n{size=-5}(Installed){/size}"

# #if mod is not installed

# else:

# $ modname = "{size=-10}" + modname + "{/size}"

#

#

# textbutton "[modname]":

# style "modmenu_select_btn"

#

# action [Hide("modmenu_mod_content"),

# Show("modmenu_mod_content",

# modid=modid,

# name=unicode(name, "utf8"),

# author=unicode(author, "utf8"),

# description=unicode(description, "utf8"),

# url=url,

# use_steam=use_steam,

# transition=dissolve),

# Play("audio", "se/sounds/open.ogg")]

#

#

# bar value YScrollValue("modselect_vp"):

# style "modmenu_select_slider"

# #yalign 0.95

Aurumbi added 4 commits February 17, 2026 13:43

Avoid CachePersonas crash

5f79ae5

Implemented cached steamhandler, To prevent crashes related to it.

639dbdc

Used steamhandler_ex in modconfig.py to fix mod browser crashing on call

2c8e315

Paged mod browser implementation

9e87fdd

muddyfish reviewed Feb 17, 2026

View reviewed changes

Aurumbi commented Feb 17, 2026

View reviewed changes

	# steammgr.CachePersonas() #TODO: Change to call the new thing...
	# steammgr.CachePersonas()


		from modloader import modconfig

		# Cache mod validity to expedite mod browser startup

Conversation

Aurumbi commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Main Changes

Fixed steamhandler's QueryApi method

Added pagination to the In-game mod browser

Load speed improvements

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Aurumbi Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Aurumbi Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Aurumbi Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Aurumbi commented Feb 17, 2026 •

edited

Loading

Fixed `steamhandler`'s `QueryApi` method

Aurumbi Feb 17, 2026 •

edited

Loading

Aurumbi Feb 17, 2026 •

edited

Loading

Aurumbi Feb 17, 2026 •

edited

Loading