Mantis Bugtracker
  

Viewing Issue Advanced Details Jump to Notes ] View Simple ] Issue History ] Print ]
ID Category Severity Reproducibility Date Submitted Last Update
0007819 [Squeak] VM feature always 07-09-14 22:54 09-01-16 19:44
Reporter lewis View Status public  
Assigned To lewis
Priority normal Resolution open Platform
Status testing   OS
Projection none   OS Version
ETA none Fixed in Version Product Version
  Product Build
Summary 0007819: Add Systemd socket activation patch for the VM in SocketPlugin
Description From: Max Leske <maxleske at gmail.com>
Date: Tue, 8 Jul 2014 16:08:01 +0200
To: Squeak Virtual Machine Development Discussion <vm-dev@lists.squeakfoundation.org>
Subject: [Vm-dev] [DEV] Systemd socket activation patch for the VM
 
Hi folks
As promised a long time ago my colleague Nik Lutz has finalized the patch for the VM that enables socket activation with Systemd. I will quickly go over what the patch is supposed to achieve, how we plan to use socket activation and what (minimal) changes are necessary.


Systemd Socket Activation:
Systemd opens a (network-) socket and waits for connections. When the first client connects the assotiated service (VM) is started. Systemd indicates via environment variable (LISTEN_FDS) that it opened a network socket.

Detailed documentation on socket activation with Systemd can be found here: http://0pointer.de/blog/projects/socket-activation.html [^]


Our scenario for socket activation:
At Cmsbox we have hundereds of images running on a server, all of which listen on a dedicated network port for incoming connections. The server is powerful enough to run all these processes concurrently, especially since not every image receives requests all the time.
With socket activation we could further lower the server load and free up resources for images with a high request rate. Images that don?t receive any requests for a specified amount of time can be suspended automatically and reactivated when a request comes in.



The VM Patch:
The attached patch tries hard to be minimal. It defines a new socket type (ProvidedTCPSocketType) which you will also find in the image (Socket>>newSystemd). ProvidedTCPSocketType is special in that it can refer to an existing TCPSocket and does not necessarily entail the creation of a new socket.


The changes in the image:
- Socket>>newSystemd tries to open a new Systemd socket.
- TCPListener>>haveSystemdAssignedPort tests if the LISTEN_FDS environment variable has been set by Systemd for the current process.
- TCPListener>>primEnvironmentAtSymbol is a utility method that we need to check for the environment variable (IMPORTANT: copied from OSProcess; uses the OSProcess plugin)
- TCPListener>>pvtNewListener: has been modified to include the check for the Systemd port. If the socket has been assigned by Systemd, use a Systemd socket, a TCP socket otherwise

Quick Howto:
- Adapt path to vm and image in pharo@.service
- Take a Seaside image and FileIn systemd.cs (tested with http://seaside.st/distributions/Seaside-2.8.4.zip) [^]
- Save as 9999.image
- Copy pharo@.service and pharo@.socket to /etc/systemd/system
- As root run:
 systemctl daemon-reload
 systemctl start pharo@9999.socket
- Open http://localhost:9999/seaside/config [^]

[- systemctl stop pharo@9999.socket]
-> Upon connection pharo@9999.socket starts: pharo@9999.service (which would start .../9999.image)

You can see this in action in the following screencast: http://youtu.be/MVPLZNKg5j8 [^]


We hope that this patch (or some version of it) will find its way into the VM. If you have any questions please feel free to ask.


Cheers,
Max (on behalf of Nik and the rest of Netstyle.ch)
Steps To Reproduce
Additional Information Posted on vm-dev:
http://lists.squeakfoundation.org/pipermail/vm-dev/2014-July/016132.html [^]

Patches are small and affect unix SocketPlugin only.
Attached Files  squeak-svn.diff [^] (1,176 bytes) 07-09-14 22:55
 systemd.cs [^] (1,679 bytes) 07-09-14 22:55
 pharo@.service [^] (396 bytes) 07-09-14 22:56
 pharo@.socket [^] (139 bytes) 07-09-14 22:56

- Relationships

- Notes
(0014507 - 37 - 37 - 37 - 37 - 37 - 37)
lewis
07-09-14 22:57

The VM changes are in squeak-svn.diff
 
(0014508 - 160 - 160 - 354 - 354 - 354 - 354)
lewis
07-10-14 00:00

Note follow up discussion http://lists.squeakfoundation.org/pipermail/vm-dev/2014-July/016153.html [^] may require change to declarations in the SocketPlugin patch.
 
(0014509 - 92 - 98 - 98 - 98 - 98 - 98)
lewis
07-10-14 11:09

Reference vm-dev discussion - final version will use this:
#define ProvidedTCPSocketType 3
 
(0014545 - 109 - 109 - 109 - 109 - 109 - 109)
maxleske
10-16-15 08:47

We're slowly rolling socket activation out to production. It would be great if this could be integrated soon.
 
(0014549 - 262 - 262 - 262 - 262 - 262 - 262)
lewis
10-20-15 23:42

Max, can you please clarify if it is OK to use a different #define ProvidedTCPSocketType? Based on the current oscog version, it would probably now need to be #define ProvidedTCPSocketType 5. It would be best to update this in the oscog branch first. Is that OK?
 
(0014550 - 24 - 24 - 24 - 24 - 24 - 24)
maxleske
10-21-15 06:39

Sure. That works for us.
 
(0014594 - 104 - 104 - 104 - 104 - 104 - 104)
maxleske
08-26-16 06:38

We're now using socket activation in production. So it would be great if this patch could be integrated.
 
(0014595 - 301 - 313 - 313 - 313 - 313 - 313)
lewis
08-28-16 16:02

The update is already present in oscog for Cog and Spur VMs, except that that ProvidedTCPSocketType is now defined as 65536, whereas the original patch as implemented for cmsbox defined it as 2. I added the update to SVN for the interpreter VM also.

Max, can you confirm if this is working for you?
 
(0014596 - 170 - 170 - 170 - 170 - 170 - 170)
maxleske
08-28-16 16:31

That's ok. Do you need me to test the integration? We're building the OpenSmalltalk VM and will use that with socket activation, so we should see possible problems there.
 
(0014597 - 130 - 130 - 130 - 130 - 130 - 130)
lewis
09-01-16 19:44

As long as it works in OpenSmalltalk VM, all is good, and if so we can close this issue. Thanks for contributing the enhancements!
 

- Issue History
Date Modified Username Field Change
07-09-14 22:54 lewis New Issue
07-09-14 22:54 lewis Status new => assigned
07-09-14 22:54 lewis Assigned To  => lewis
07-09-14 22:55 lewis File Added: squeak-svn.diff
07-09-14 22:55 lewis File Added: systemd.cs
07-09-14 22:56 lewis File Added: pharo@.service
07-09-14 22:56 lewis File Added: pharo@.socket
07-09-14 22:57 lewis Note Added: 0014507
07-10-14 00:00 lewis Note Added: 0014508
07-10-14 11:09 lewis Note Added: 0014509
10-16-15 08:45 maxleske Issue Monitored: maxleske
10-16-15 08:47 maxleske Note Added: 0014545
10-20-15 23:42 lewis Note Added: 0014549
10-21-15 06:39 maxleske Note Added: 0014550
08-26-16 06:38 maxleske Note Added: 0014594
08-28-16 16:02 lewis Note Added: 0014595
08-28-16 16:02 lewis Status assigned => testing
08-28-16 16:31 maxleske Note Added: 0014596
09-01-16 19:44 lewis Note Added: 0014597


Mantis 1.0.8[^]
Copyright © 2000 - 2007 Mantis Group
81 total queries executed.
47 unique queries executed.
Powered by Mantis Bugtracker