clsim - propagation server

[David Schultz]

Melanie has finally found a need for a "clsim server," where you can send it things to propagate and it will return the results.

The problem:

• Muongun results in really, really large files, and the only way to reduce filesize is to run through detector simulation (to throw out things that don't trigger, I guess).
• This means running lots of cpu modules on a gpu machine, which is very inefficient.
• Worse, these are often memory-intensive; gpu jobs should not be using large amounts of memory in the current NPX setup.

Solution:

• Run the job on a regular cpu slot, with a remote call to a clsim server for the photon propagation.

Implementation:

• Prototype: just send whole frame to the gpu machine, with it returning the completed frame.
• Better: send bunches of photons, return raw results
  • helps for high energy events where a single frame is huge
• RPC communication: json over http? ZeroMQ?
• Choosing the gpu machine: need a central distribution server
  • registers gpu servers and can load balance client requests
  • if gpu server is open to world, can give client its address to push work directly
  • if gpu server is behind firewall, can act as intermediary for pull-based connections

Control flow diagram (direct):

Client I3Module     Central Server       GPU Server
       |                   |      register    |
       |      request      | <--------------- |
       | ----------------> |                  |
       |    try direct     |                  |
       | <---------------- |                  |
       |        data       |                  |
       | -----------------------------------> |
       |                   |      result      |
       | <----------------------------------- |
       |      release      |                  |
       | ----------------> |                  |
       |                   |      remove      |
       |                   | <--------------- |

Control flow diagram (intermediary):

Client I3Module     Central Server       GPU Server
       |                   |      register    |
       |      request      | <--------------- |
       | ----------------> |                  |
       |     indirect      |                  |
       | <---------------- |                  |
       |       data        |                  |
       | ----------------> |                  |
       |                   |       poll       |
       |                   | <--------------- |
       |                   |       data       |
       |                   | ---------------> |
       |                   |      result      |
       |      result       | <--------------- |
       | <---------------- |                  |
       |      release      |                  |
       | ----------------> |                  |
       |                   |      remove      |
       |                   | <--------------- |

[Alex Olivas]

Is there simply a need/use or is this critical? There are mixed priorities in this ticket. There was talk back in the day about doing the same with photonics, but that never really got much traction. A "need" was identified, but it remained low priority.

Is this part of simulation or possibly an iceprod 2 feature?

[David Schultz]

This is critical in the sense that Melanie needs this soon or she'll be kicked off the gpus in Madison. We haven't done muongun much in production, but we should in the future. So production will likely encounter a similar issue as well.

This is more part of simulation than iceprod.

[David Schultz]

Prototype for direct communication finished in r129456/IceCube. Uses ZeroMQ to send the frame hierarchy (GCDQ) to the gpu, and get the resultant Q frame back.

Note that Geant4 doesn't play very nice with this, for some reason. Or maybe that's just clsim + Geant4 in general? But compiling without Geant4 makes it work.

[David Schultz]

The distribute project by Claudio does this and a lot more. Defer to that, and close this out.

[Alex Olivas]

Milestone IceSim? 6 deleted