Skip to content

Try to make lupin working with threads#4333

Open
samuelgarcia wants to merge 22 commits intoSpikeInterface:mainfrom
samuelgarcia:thread_an_no_nogil
Open

Try to make lupin working with threads#4333
samuelgarcia wants to merge 22 commits intoSpikeInterface:mainfrom
samuelgarcia:thread_an_no_nogil

Conversation

@samuelgarcia
Copy link
Member

This try to make lupin working with thread.
This will be usefull because on windows the spwan and process parrallel is using too much memory

  • Release on numba kernel
  • avoid racing when compilation on numba
  • and some other trick
  • fix mp_context and Lock

this replace:
#4331
#4309

@alejoe91 alejoe91 added the sorters Related to sorters module label Jan 20, 2026
@yger
Copy link
Collaborator

yger commented Jan 26, 2026

I have a problem with the branch @samuelgarcia , this is crashing for your dev notebooks with the compute_analyzer() case...

@samuelgarcia samuelgarcia added this to the 0.104.0 milestone Feb 3, 2026
Comment on lines 434 to 438
if preferred_mp_context is not None and preferred_mp_context != mp_context:
warnings.warn(
f"You processing chain using pool_engine='process' and mp_context='{mp_context}' is not possible."
f"So use mp_context='{preferred_mp_context}' instead")
mp_context = preferred_mp_context
Copy link
Member

@alejoe91 alejoe91 Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if a preprocessor wants to use spawn on linux (maybe because it uses GPUs)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then this is handle because mp_context = preferred_mp_context
This is the same logic as before ?

@yger
Copy link
Collaborator

yger commented Feb 5, 2026

@sam you should merge my optimizations for the waveforms/templates estimations. And do not forget your patch for shallow_copies.

@samuelgarcia
Copy link
Member Author

Ready on my side

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sorters Related to sorters module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants