Pull Requests

#172 - Add proper iterator for stream interface and add transaction class for memory interface
#152 - New EPICs interface
#158 - New build system and updated readme files
#143 - Packetizer Version 2 support
#184 - Add locking token for transactions and frame receivers
#163 - Added new axi hardware
#167 - Addresses ESROGUE-176 & RSSI Errors
#169 - Add list variable support in EPICS & LocalVariable
#164 - Added UDP Server
#176 - Support for 64 bit and 128 bit PRBS
#154 - Remove max buffer size from frame request API
#165 - Add support for RSSI server.
#178 - Generate address space overlap warnings
#159 - Fix SSI Default and Error Handling In DataCard and AxiStream Interface
#181 - Enable bit sizes larger than 32 as strings, fix epics string handling
#179 - Added key/value support to yaml and speed up root updates
#189 - Updated release notes
#173 - Allow Hub.doTransaction() to be overridden in Python
#161 - Added sprintf style creator for general error
#166 - Add retry for rawRead and rawWrite
#180 - Deprecate class method function command decorators
#187 - Support signed ints which are not multiples of 8-bits
#174 - Variable default value enhancements
#175 - Allow Device enables to depend on other Variables
#153 - Move smem, simulation and mysql to interfaces directory
#183 - Add version attribute with the module version number
#188 - Fix bad vector iterator in memory Slave transaction tracking
#182 - Set flags in outbound frame for packetizer
#162 - Added error generation for bad sizes when using PrbsTx
#150 - Large blocks
#177 - Added size checking, fix block overlap check
#170 - Fix bug in pyrogue/protocols/_Network.py and improve setup script.
#168 - Add parsing support for lists and dicts held by LocalVariable

Pull Request Details

Add proper iterator for stream interface and add transaction class for memory interface


Author:	Ryan Herbst [email protected]
Date:	Wed Mar 14 11:24:21 2018 -0700
Pull:	#172 (1972 additions, 1368 deletions, 50 files changed)
Branch:	slaclab/ESROGUE-179
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-179

Notes:

Stream Iterator

This first change in this pull request modifies the Frame/Buffer interface to use proper iterators. The existing frame->getCount() and frame->getBuffer() are now replaced with frame->beginBuffer() and frame->endBuffer() iterators. The previous iterator model for frames (startRead, startWrite, nextRead, nextWrite) have been removed in favor of a proper frame iterator in the form of frame->begin() and frame->end().

The buffer interface to data is now also consistent with the iterator model allowing the user to access buffer bytes in the following ways:
// for payload data space
for (Buffer::iterator it = buffer->begin(); it != buffer->endPayload(); ++it)  
// for all frame data space
for (Buffer::iterator it = buffer->begin(); it != buffer->end(); ++it) 
Users can choose to iterate directly through the entire frame byte array in the following ways:
// for frame payload data space
for (Frame::iterator it = frame->beginRead(); it != frame->endRead(); ++it)  
// for all frame data space
for (Frame::iterator it = frame->beginWrite(); it != frame->endWrite(); ++it) 
Alternatively a user can iterate over buffers and treat the buffer data as large blocks:
for(Frame::BufferIterator it = frame->beginBuffer(); it != frame->endBuffer(); ++it)
   write(socket, (*it)->begin(), (*it)->endPayload()-(*it)->begin());  
Both the frame and buffer have cleaner methods to get and adjust frame lengths. The python interface to frame data does not change.

Memory Transaction Class

The second change is a major modification to the memory interface. A new class type Transaction is created to contain the transaction information as well as provide the interface to moving data and completing the transaction. This new class is passed between the master and slave to initiate a memory transaction. The slave then operates on this transaction object to get and put data and complete the transaction. Right now this transaction wraps a char * iterator, but in the future more complex iterators will be added as needed by more complicated memory transaction masters.

Both python and c++ memory slave code requires updates to work with this new model. Python memory slaves have a cleaner interface now when interfacing with the Transaction object.

New EPICs interface


Author:	Ryan Herbst [email protected]
Date:	Mon Feb 26 10:26:50 2018 -0800
Pull:	#152 (2088 additions, 268 deletions, 27 files changed)
Branch:	slaclab/ESROGUE-141
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-141

Notes:

The API is the same, with the addition of a slave interface to map an epics variable to a rogue stream.

New build system and updated readme files


Author:	Ryan Herbst [email protected]
Date:	Fri Mar 2 13:14:31 2018 -0800
Pull:	#158 (957 additions, 411 deletions, 46 files changed)
Branch:	slaclab/ESROGUE-168
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-168

Notes:

The new build system uses cmake. This system will auto detect the packages and compile options required on the various machines. I have tested on archlinux, rhel6 and ubuntu in windows.

I have also updated the readme files combining them into two. The top level Readme.md covers download, build, install and the support packages. Please check the list of python3 modules to make sure it is up to date.

The second Readme_local_packages.md is for the poor soulds that need to build the support libraries which should not be necessary for any machines other than slac.

I have also created 2 sets of setup scripts. One for rogue and one for the slac environment.

Packetizer Version 2 support


Author:	Ryan Herbst [email protected]
Date:	Mon Mar 5 10:35:25 2018 -0800
Pull:	#143 (935 additions, 208 deletions, 14 files changed)
Branch:	slaclab/ESROGUE-153
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-153

Notes:

Add locking token for transactions and frame receivers


Author:	Ryan Herbst [email protected]
Date:	Tue Mar 20 12:14:30 2018 -0700
Pull:	#184 (583 additions, 243 deletions, 39 files changed)
Branch:	slaclab/ESROGUE-194
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-194

Notes:

I have created two classes, FrameLock and TransactionLock which are used to lock Frame data and Transaction data respectively. The transaction lock was always required, but in a less convenient form, and adding a Frame data lock allows us to make less assumptions on how Frame data is processed and stored, opening up the possibility of frame data queues.

The lock can be gained in a c++ process:
ris::FrameLockPtr flock = frame->lock()
rim::TransactionLockPtr tlock = transaction->lock()
or in python:
flock = frame.lock()
tlock = transaction.lock()
The lock will be removed when the lock container goes out of scope. Both lock classes have an unlock and lock method for turning the lock off and on as needed. Both transaction and frame locking is done properly in internal classes now.

Added new axi hardware


Author:	Ryan Herbst [email protected]
Date:	Fri Mar 2 19:23:03 2018 -0800
Pull:	#163 (787 additions, 2 deletions, 15 files changed)
Branch:	slaclab/ESROGUE-184
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-184

Notes:

hardware/data/DataCard and hardware/rce/AxiStream are replaced by hardware/axi/AxiStreamDma
hardware/data/DataMap and hardware/rce/MapMemory are replaced by hardware/axi/AxiMemMap

The data and rce interfaces are now deprecated and will be removed in a future version of rogue.

Addresses ESROGUE-176 & RSSI Errors


Author:	Ryan Herbst [email protected]
Date:	Thu Mar 8 00:21:49 2018 -0800
Pull:	#167 (256 additions, 230 deletions, 19 files changed)
Branch:	slaclab/ESROGUE-176
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-176

Notes:

This pull request address silent corruption of packetizer/RSSI data due to overflow on UDP receive. This pull request also simplifies the buffer size settings in pyrogue/protocols/_Network. The new interface has the user set the jumbo frame mode for the UDP engine. The UDP engine then provides the max payload size which can be passed as the segment size to the RSSI engine. The packetizer no longer takes a segment size arg as it is not necessary and is controlled by the buffer sizes provided by the RSSI engine.

Add list variable support in EPICS & LocalVariable


Author:	Ryan Herbst [email protected]
Date:	Fri Mar 9 08:41:12 2018 -0800
Pull:	#169 (264 additions, 113 deletions, 9 files changed)
Branch:	slaclab/ESROGUE-190
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-190

Notes:

List types are now supported for LocalVariables. The LocalBlock set and get use the copy function to avoid locking issues at the expense of extra copies.

Sets from the rogue side show up as the proper length for monitored PVs (camonitor) but the max size will always be returned for caget. I am not sure how to solve this yet.

Puts to rogue will always show up as the correct size in rogue.

Long string support is not in place with this version.

Added UDP Server


Author:	Ryan Herbst [email protected]
Date:	Fri Mar 2 19:23:22 2018 -0800
Pull:	#164 (353 additions, 6 deletions, 5 files changed)
Branch:	slaclab/ESROGUE-187
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-187

Notes:

Support for 64 bit and 128 bit PRBS


Author:	Ryan Herbst [email protected]
Date:	Thu Mar 15 09:43:04 2018 -0700
Pull:	#176 (147 additions, 125 deletions, 3 files changed)
Branch:	slaclab/ESROGUE-154
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-154

Notes:

This pull requests adds support for 64-bit and 128 bit PRBS data streams.

Byte width and taps may be updated after the PRBS object is created using the following methods:
setWidth(uint32_t size)
setTaps(uint32_t count, uint8_t * taps)
In python the setTaps method takes a byte array of tap values.

One note. The PRBS stream expects the frame sequence # and length values to roll over at 32-bits.

Remove max buffer size from frame request API


Author:	Ryan Herbst [email protected]
Date:	Mon Feb 26 12:55:42 2018 -0800
Pull:	#154 (62 additions, 112 deletions, 36 files changed)
Branch:	slaclab/ESROGUE-167
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-167

Notes:

This will break some frame transmitters, but not many exist in python right now.

Turns out including the maxBufferSize field is unnecessary. If a module needs a smaller buffers it should request smaller frames downstream and recombine the buffers into an upstream frame. The interface is confusing as is.

I ended up doing this in the packetizer and it makes more sense.

Add support for RSSI server.


Author:	Ryan Herbst [email protected]
Date:	Tue Mar 6 09:53:48 2018 -0800
Pull:	#165 (91 additions, 52 deletions, 6 files changed)
Branch:	slaclab/ESROGUE-188
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-188

Notes:

Existing RSSI server class was not fully supported. This pull request adds the necessary code for a server RSSI to work. This pull request also re-enables the frame allocate transport interface in the RSSI and packetizer , which allows these modules to be connected back to back for testing. Additional debugging in RSSI is also added.

Generate address space overlap warnings


Author:	Ryan Herbst [email protected]
Date:	Thu Mar 15 21:05:23 2018 -0700
Pull:	#178 (103 additions, 3 deletions, 10 files changed)
Branch:	slaclab/ESROGUE-52
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-52

Notes:

This pull request adds two features.

The first looks for overlapping variables within a block. This is done by doing a bit by bit check as variables are added to the block. A critical log message is generated when an overlap is detected.

The second feature looks for overlapping device address spaces. In order for this feature to work properly I added a unique ID to each memory interface slave. The device will query the memBase chain to find the ID of the memory interface it is connected to. This allows the pyrogue root to avoid mislabeling overlaps between devices connected to different memory interfaces. If any devices overlap and are connected to the same memory interface, a critical log message is generated.

Fix SSI Default and Error Handling In DataCard and AxiStream Interface


Author:	Ryan Herbst [email protected]
Date:	Fri Mar 2 10:03:24 2018 -0800
Pull:	#159 (74 additions, 31 deletions, 3 files changed)
Branch:	slaclab/ESROGUE-182
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-182

Notes:

Enable bit sizes larger than 32 as strings, fix epics string handling


Author:	Ryan Herbst [email protected]
Date:	Mon Mar 19 12:51:17 2018 -0700
Pull:	#181 (54 additions, 29 deletions, 3 files changed)
Branch:	slaclab/epics-dev

Notes:

This pulls request includes:

All values greater than 32bit should be treated as string, not only 64bit values (The GitHash for example is 160bit long and it was not treated correctly before).

Also, as EPICS string PV are limited to 40 chars, I propose to use array of char instead. On the client side, the user must convert the array of char to a string. caget and caput utilities have the -S option for this purpose. Before this change, long string (like the buildstamp) or long numbers (as the githash on hex representation) were truncated to 40 chars. With this modification the values are read correctly, for example:
[jvasquez@tid-pc93099 ~]$ caget -S jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:GitHash
jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:GitHash 0xb8669ccf9c72e481e6c81870111ffc5206c7640
[jvasquez@tid-pc93099 ~]$ caget -S jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:BuildStamp
jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:BuildStamp CryoDetCmbHcdBpEth: Vivado v2016.4, rdsrv223 (x86_64), Built Thu Nov 30 15:51:15 PST 2017 by mdewart
It also works for reading/wrtting string:
[jvasquez@tid-pc93099 ~]$ caget -S jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:LocalString
jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:LocalString Local variable with a string
[jvasquez@tid-pc93099 ~]$ caput -S jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:LocalString "New string set from EPICS"
Old : jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:LocalString Local variable with a string
New : jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:LocalString New string set from EPICS
[jvasquez@tid-pc93099 ~]$ caget -S jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:LocalString
jesus_test:AMCc:FpgaTopLevel:AmcCarrierCore:AxiVersion:LocalString New string set from EPICS

Added key/value support to yaml and speed up root updates


Author:	Ryan Herbst [email protected]
Date:	Tue Mar 20 08:31:41 2018 -0700
Pull:	#179 (17 additions, 47 deletions, 3 files changed)
Branch:	slaclab/ESROGUE-192
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-192

Notes:

This pull request address the slow variable read/write access caused by generating streaming yaml configuration updates from the pyrogue root. The fix is to avoid generating a dictionary based yaml configuration stream, instead generating key/value pairs such as:
testRoot.Fpga.AxiVersion.ScratchPad:0x55
Before this change the variable access rate over RSSI was 2Khz. After this update the var access rate over RSSI is 6Khz, compared to the raw mode access at 8Khz. I removed the update arg from the Variables since it no longer is needed to help this problem and only adds confusion.

In order to support using these yaml dumps, I updated the setYaml function to properly process yaml files using the above key/value pair style.

Updated release notes


Author:	Ryan Herbst [email protected]
Date:	Wed Mar 21 11:20:43 2018 -0700
Pull:	#189 (30 additions, 29 deletions, 1 files changed)
Branch:	slaclab/rel-notes

Notes:

Allow Hub.doTransaction() to be overridden in Python


Author:	Ryan Herbst [email protected]
Date:	Wed Mar 14 11:36:43 2018 -0700
Pull:	#173 (50 additions, 7 deletions, 3 files changed)
Branch:	slaclab/hub-wrap

Notes:

A pyrogue Device can now override _doTransaction() to perform a limited amount of address manipulation.

Added sprintf style creator for general error


Author:	Ryan Herbst [email protected]
Date:	Fri Mar 2 13:07:17 2018 -0800
Pull:	#161 (34 additions, 17 deletions, 3 files changed)
Branch:	slaclab/ESROGUE-186
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-186

Notes:

Add retry for rawRead and rawWrite


Author:	Ryan Herbst [email protected]
Date:	Tue Mar 6 09:54:05 2018 -0800
Pull:	#166 (20 additions, 17 deletions, 1 files changed)
Branch:	slaclab/ESROGUE-162
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-162

Notes:

This is not yet tested. I don't have a setup to try the rawRead / rawWrite

Deprecate class method function command decorators


Author:	Ryan Herbst [email protected]
Date:	Tue Mar 20 08:32:02 2018 -0700
Pull:	#180 (28 additions, 8 deletions, 2 files changed)
Branch:	slaclab/ESROGUE-165
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-165

Notes:

Adding @pyrogue.Command decorators on class methods will no longer be supported in a future release. This PR adds a deprecation message and modified the Root class to no longer use this decorator.

Using decorators on functions within the Device's init method is the preferred way to create Commands attached to Device functions.

Support signed ints which are not multiples of 8-bits


Author:	Ryan Herbst [email protected]
Date:	Wed Mar 21 10:10:39 2018 -0700
Pull:	#187 (24 additions, 10 deletions, 2 files changed)
Branch:	slaclab/ESCRYODET-110
Jira:	https://jira.slac.stanford.edu/issues/ESCRYODET-110

Notes:

Variable default value enhancements


Author:	Ryan Herbst [email protected]
Date:	Wed Mar 14 11:25:49 2018 -0700
Pull:	#174 (27 additions, 3 deletions, 3 files changed)
Branch:	slaclab/block-default

Notes:

Fixed RemoteVariable initial/default values

Can now also give a Device a dict of default values for its Variables

Allow Device enables to depend on other Variables


Author:	Ryan Herbst [email protected]
Date:	Wed Mar 14 11:24:38 2018 -0700
Pull:	#175 (17 additions, 5 deletions, 1 files changed)
Branch:	slaclab/enable-chain

Notes:

A Device enable is already implicitly dependent on it's parent's enable. This extends that concept to allowed the enable to depend on one or more other variables in the tree.

This allows e.g. a Device on a remote link to only be enabled once the link is up.

Move smem, simulation and mysql to interfaces directory


Author:	Ryan Herbst [email protected]
Date:	Mon Feb 26 13:00:02 2018 -0800
Pull:	#153 (13 additions, 7 deletions, 5 files changed)
Branch:	slaclab/ESROGUE-164_2
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-164_2

Notes:

I also moved protocols.py to protocols/_Network.py and auto included from protocols/init.py

Usage of UdpRssiPack won't change.

Add version attribute with the module version number


Author:	Ryan Herbst [email protected]
Date:	Mon Mar 19 15:36:07 2018 -0700
Pull:	#183 (19 additions, 0 deletions, 3 files changed)
Branch:	slaclab/add-version-number

Notes:

Add the __version__ attribute with the version number. When importing pyroguethe module one can call pyrogue.__version__ and importing other modules conditionally. An example is the EPICS module which is different in the current release and the after the pre-release branch will be merge.

Fix bad vector iterator in memory Slave transaction tracking


Author:	Ryan Herbst [email protected]
Date:	Wed Mar 21 08:18:57 2018 -0700
Pull:	#188 (9 additions, 8 deletions, 3 files changed)
Branch:	slaclab/ESROGUE-196
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-196

Notes:

Also includes cleanup of redundant GIL release in epics Slave interface and cleans up transaction deletion in SrpV3 to match SrpV0.

Set flags in outbound frame for packetizer


Author:	Ryan Herbst [email protected]
Date:	Tue Mar 20 08:31:17 2018 -0700
Pull:	#182 (11 additions, 6 deletions, 2 files changed)
Branch:	slaclab/packetizer_fix

Notes:

When doing another update I noticed that the flags were being modified in the inbound frame instead of the outbound frame.

Added error generation for bad sizes when using PrbsTx


Author:	Ryan Herbst [email protected]
Date:	Fri Mar 2 12:23:55 2018 -0800
Pull:	#162 (10 additions, 7 deletions, 1 files changed)
Branch:	slaclab/ESROGUE-180
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-180

Notes:

Large blocks


Author:	Ryan Herbst [email protected]
Date:	Mon Feb 26 10:26:12 2018 -0800
Pull:	#150 (11 additions, 3 deletions, 1 files changed)
Branch:	slaclab/large_blocks

Notes:
Support for the creation of larger blocks in a device then the min size.

Added size checking, fix block overlap check


Author:	Ryan Herbst [email protected]
Date:	Thu Mar 15 09:43:19 2018 -0700
Pull:	#177 (11 additions, 2 deletions, 1 files changed)
Branch:	slaclab/ESROGUE-85
Jira:	https://jira.slac.stanford.edu/issues/ESROGUE-85

Notes:

Blocks rawWrite and rawRead accesses out of the defined size of a Device. The size of the device can be set at creation and/or automatically detected by the variables it contains.

Fix bug in pyrogue/protocols/_Network.py and improve setup script.


Author:	Ryan Herbst [email protected]
Date:	Mon Mar 12 15:49:00 2018 -0700
Pull:	#170 (2 additions, 2 deletions, 2 files changed)
Branch:	slaclab/epics-dev

Notes:

Fix bash setup script: Generate absolute TOP directory path

Make UdpRssiPack backward compatible setting packVer default to 1

Add parsing support for lists and dicts held by LocalVariable


Author:	Benjamin Reese [email protected]
Date:	Wed Mar 7 12:55:13 2018 -0800
Pull:	#168 (2 additions, 0 deletions, 1 files changed)
Branch:	slaclab/LocalVariable-list

Release 2.7.0

Pull Requests

Pull Request Details

Add proper iterator for stream interface and add transaction class for memory interface

Stream Iterator

Memory Transaction Class

New EPICs interface

New build system and updated readme files

Packetizer Version 2 support

Notes:

Add locking token for transactions and frame receivers

Added new axi hardware

Notes:

The data and rce interfaces are now deprecated and will be removed in a future version of rogue.

Addresses ESROGUE-176 & RSSI Errors

Add list variable support in EPICS & LocalVariable

Added UDP Server

Notes:

Support for 64 bit and 128 bit PRBS

Remove max buffer size from frame request API

Add support for RSSI server.

Generate address space overlap warnings

Fix SSI Default and Error Handling In DataCard and AxiStream Interface

Notes:

Enable bit sizes larger than 32 as strings, fix epics string handling

Added key/value support to yaml and speed up root updates

Updated release notes

Notes:

Allow Hub.doTransaction() to be overridden in Python

Added sprintf style creator for general error

Notes:

Add retry for rawRead and rawWrite

Deprecate class method function command decorators

Support signed ints which are not multiples of 8-bits

Notes:

Variable default value enhancements

Allow Device enables to depend on other Variables

Move smem, simulation and mysql to interfaces directory

Add version attribute with the module version number

Fix bad vector iterator in memory Slave transaction tracking

Set flags in outbound frame for packetizer

Added error generation for bad sizes when using PrbsTx

Notes:

Large blocks

Notes: Support for the creation of larger blocks in a device then the min size.

Added size checking, fix block overlap check

Fix bug in pyrogue/protocols/_Network.py and improve setup script.

Add parsing support for lists and dicts held by LocalVariable

Notes:

Notes:
Support for the creation of larger blocks in a device then the min size.