[photospline] Merge photospline2 branches into the trunk

[Alex Olivas]

In the works are improvements to the interface, speed-ups, and reduce memory footprint by allowing shared resources. This will require adaptations to photonics-service as well.

[Christopher Weaver]

The new library code is at ​https://github.com/cnweaver/photospline/.

[David Schultz]

I'd really, really like this to work without dedicated pilot/cleanup jobs and have them be built-in.

jvs: "what you basically want is a reference-counted singleton in shared memory. this should be straightforward, but the creation phase needs to be guarded somewhat carefully"

[Jakob van Santen]

It should be noted that the new library is also called "photospline" and so cannot coëxist with the libphotospline from the photospline I3 project. When we switch from project to tool, we will also have to switch photonics-service and MuonGun?.

photonics-service:

• The beginnings of a port are in photonics-service/branches/photospline-2.0. This exists mostly to ensure that sampling time delays works as expected.
• I3PhotoSplineTable exists mostly to store table metadata and provide the Eval() function that wraps tablesearchcenters() and ndsplineeval(). Since photospline::splinetable provides this directly, I3PhotoSplineTable can probably just be removed.
• I3PhotoSplineService needs to learn how to allocate (or find) tables in shared memory, and clean up after itself. The docs for ​boost::interprocess::unique_instance seem to imply that atomic creation and destruction are supported out of the box, and there is also an ​interprocess shared_ptr that can be used to count references. Ideally we'd also remove the entire shared memory segment when the reference count reaches zero, but it's not clear how to do that without freeing mutex data that is still in use.

MuonGun?:

• MuonGun? has its own SplineTable class that exists only to support serialization. It should be easy to convert this to the new library.

Are there any other projects that secretly use photospline?

[Christopher Weaver]

sim-services/I3CrossSection.h is another use of photospline which I just added recently (and should be trivial to port).
Out-of-tree but somewhat widely used projects which are affected are AtmosphericSelfVeto? and NewNuFlux?.

[Jakob van Santen]

In 153994/IceCube:

Enable shared memory storage for I3PhotoSplineService

Tables are stored in a shared memory segment named after the table paths that
is created on demand. The segment also maintains a reference count, and the
I3PhotoSplineService dtor frees the segment when it drops to zero. See #1962.

[Jakob van Santen]

In 153998/IceCube:

Fix real and potential issues with memory shared between processes of different
users. See #1962.

• Put string contents in shared storage
• Properly protect against double convolution
• Use unique segment names for each user

There is still an issue with the way this relies on the I3PhotoSplineTable dtor
to decrement the reference count and free resources: the destructor is not run
after a process is sent SIGKILL. In the normal case this leads to a dangling
reference and the segment never being unlinked; in the worst case this could
happen while the process holds the segment's mutex, leaving it locked for all
time.

[Alex Olivas]

MuonGun? still needs to be ported to use photospline 2.

[Kevin Meagher]

The following is a list of projects which currently use photospline and need to be changed to the new version:

• MuonGun
• NewNuFlux
• photonics-service
• sim-services

[Alex Olivas]

Olivas: sim-services

[Alex Olivas]

Metaproject branch is compiling and linking: ​http://code.icecube.wisc.edu/svn/meta-projects/combo/branches/ps2

There are a couple more project branches we need to merge and/or update.

[Alex Olivas]

photonics-service tests are crashing when using the branch. This isn't likely to be ready for this round of production. Pushing the milestone.

[Kai Krings]

I reviewed the changes that were made to the SplineTable class in MuonGun and to the I3CrossSection class in phys-services. I replaced all raw pointers to spline tables with shared pointers (plus some more minor changes). While doing that I found a bug in the equal operator of the splinetable class in photospline (due to a failing MuonGun unit test, which uses this operator). The bug is fixed in the current trunk of photospline.

[Christopher Weaver]

If the fixed photospline trunk seems to be doing what's needed please ping me if/when you would like a patch release.

[Kai Krings]

@cweaver: Thanks, I'll contact you once I've solved the issue with two failing unit tests in photonics-service, although I think the problem is rather in I3PhotoSplineService than in photospline itself.

[Kai Krings]

I would also like to discuss the idea of modernizing photonics-service by at least replacing the used raw pointers and C-arrays. I would also volunteer to do this.

[Jakob van Santen]

Replying to kkrings:

I would also like to discuss the idea of modernizing photonics-service by at least replacing the used raw pointers and C-arrays. I would also volunteer to do this.

That's a good idea. The raw pointers were there to minimize copies when crossing language barriers between C, C++, and Python. With photospline 2 and boost-numpy, it should be much easier to do this cleanly and safely.

[Alex Olivas]

Milestone Winter Solstice 2019 deleted

[Alex Olivas]

Milestone Vernal Equinox 2020 deleted