Skip to content

DIOT #84

@jordens

Description

@jordens

Since the topic comes up frequently and I couldn't find a place to tack this onto, here is an unsorted collection of things we noticed while considering moving away from EEM towards DIOT crates.

Problems we're facing with the current legacy Sinara EEM style:

  • Adding/removing/swapping cards requires disassembly of crate (hassle, risk, cost, usability limitation, and frustration for users). The ribbon cables don't have proper cable management and with more than a few it becomes dangerous/impossible to pull out and insert cards.
  • No hot-plugging: hot plugging happens by accident and is disastrous
  • Ribbon cables/IDC connectors have ill-defined impedance, lots of crosstalk, layered ribbon cables on Kasli defeat G+-G shielding topology, bad SI
  • Ribbon cables block airflow
  • Ribbon cables require large force to install/remove, can not be easily inserted/removed with modules in crate
  • IDC connectors have limited durability
  • Uniform standard ribbon cable length is too long for near modules requiring cable folding, increased crosstalk, air flow blockage, but too short with cards far away leading to tensioning
  • IDC connectors are physically large and occupy lots of board area
  • EEM 8/9 are offset from 0-7,10,11 leading to folding, uneven strain
  • IDC strain relief can't be used on 4 HP modules: fragile
  • A single ribbon cable over one non-strain-relief IDC connector is the limit to 4 HP width
  • Ribbon cables risk being scratched/worn by adjacent PCB (during installation and insertion/removal attempts)
  • Need to tie down ribbon cables with cable ties (ties need to be redone on each card change)
  • Standard ribbon cables are manufactured with both connectors pointing the same direction, not opposite as EEM requires: leads to manual and error-prone re-assembly of the ribbon cable with one side strain-relief to flip orientation
  • Very limited power through barrel connector and ribbon cables
  • Unspecified power envelope
  • Power through front panel requires elaborate EMI insolation and chokes at each egress/ingress
  • Ambiguous flexibility: unclear whether unused barrel connections can be used to power addtl devices
  • No slot monitoring of power, presence, health, identification
  • Immediate power to all modules is dangerous to FPGA pins, violating sequencing
  • No power switch: re-plugging barrel connector to cycle is risky, causes surge and wear
  • Lack of power sequencing causes surge
  • Barrel connectors easily come undone, screw connectors aren't standard
  • Powering flow is error-prone/ambiguous/dangerous: see Clocker/Kasli/Stabilizer/Banker/FastServo/HVAMP with PoE plus front and rear barrel/molex connector plus EEM power in all possible configurations and corner-cases, it's unclear where to power a crate
  • No well-defined clock distribution
  • MMCX only supplies 4 modules
  • clocking requires MMCX cables/tie-downs/insertion/removal hassle, connector wear, risk to connector damage
  • MMCX connector positions (Kasli/Kasli-SoC) are awkward but no space elsewhere
  • Since well-defined clock distribution can not be relied on, reference clocking options become an afterthought (Pounder, Fast Servo etc)
  • Standard MMCX cables are manufactured to have both connectors pointing the same direction, not opposite as EEM requires: leads to strain and twist of the MMCX cable
  • Kasli-SoC, Fast-Servo can't insert/swap SD card without disassembly of crate
  • 4 HP is too small for FPGA fans (Kasli, Kasli-SoC, Phaser) and too small for 8 SMA
  • Only 8 LVDS pairs per EEM connector leading to undesirable trade-offs (Urukul, Sampler, Grabber)
  • Connecting an unused EEM cable (EEM1 on Sampler) often leads to unexpected errors (LVDS FS)
  • Renumbering to increase/decrease EEM count on a module usually requires complete renumbering/reassembly/replugging of all modules
  • One-EEM vs two-EEM vs three-EEM is error-prone (DIP switches, LVDS fail safe), ground/power loops, unclear module identification and powering, choice overwhelms user
  • Mezzanines (Stabilizer, Mirny etc) are fragile, hard to debug, and electrically inferior
  • Many modules require forced air cooling (Kasli, Kasli-SoC, Phaser, HVAMP32, HVAMP-8CH etc) but the crate doesn't specify or provide any, leading to ad-hoc inferior solutions (small fans are noisy, fragile, unmonitored, uncontrolled, unreliable, have little mechanical compatibility for mounting), horizontal flow fans are against crate design, c.f. Kasli, Phaser fan/thermal design issues, adjacent 4 HP module blocks fan airflow, incompatible with mezzanine presence
  • Proper convective cooling or fans require spacing out the modules, complicating crate assembly and disassembly/module changes: requires additional/moved rails, additional blank front panels (e.g. 4 HP Phaser is useless in practice), fan mgmt, fan EMI handling
  • Space inefficient: due to cooling requirements, a crate with one Kasli and several modules is often not full (space-wise), but rarely empty enough to insert another Kasli with modules
  • Convective cooling holes in crate habitually blocked in rack installation
  • Mezzanine electrical and mechanical design is poor (no IDC spacer with correct length, nuts on carrier underside thicker than allowed, no reliable IDC connector alignment, IDC has bad SI, electrical interface undefined)
  • Mezzanine usage often requires different front panel (Almazny), removing some of the mezzanine flexibility advantage, leading to discarded front panels
  • JTAG/SWD connectors in varying positions with reduced accessibility (Kasli, Stabilizer) requiring crate disassembly
  • Use case prioritization is unclear and leads to inefficient designs/prioritization: e.g. Stabilizer-Kasli/Urukul EEM connectivity vs networked autonomy
  • Dumb modules (Clocker, Stabilizer, FastServo, HVAMP-8CH, Thermostat-EEM, ...) waste an EEM slot+cable just to be powered
  • EEM was not designed to last. It was a convenient quick-fix when we needed
    something working quickly. It has zero support or synergy outside Sinara.
  • No certification (CE etc)

DIOT downsides:

  • More expensive (backplane, connectors, receptacles vs IDC cables): would conceivably increase initial cost of a typical crate by 10% (? TBC)
  • Only 8 peripherals per crate (compared to max 12 in theory per Kasli with EEM)
  • Fixed board length (220 mm): no short DIO-BNC/SMA boards
  • Fixed board width (6 HP): no 8x BNC DIO or Sampler-BNC, no 4+4 carrier+mezzanine
  • Debugging/deployment/direct JTAG-flashing a card requires an adapter/extender (Kasli, Kasli-SoC)
  • Pounder/Driver mezzanines would need to become (a) RTMs, or (b) be consolidated with Stabilizer or (c) made thinner to fit into 6 HP (like DIOT/cPCI-S/FMC mezzanines): otherwise waste a slot
  • Thermostat controlling Zotino DAC would need to be an RTM (cabling), or become a thinner mezzanine to fit 6 HP, or be merged
  • Almazny redesign to fit 6 HP
  • Clocker reevaluation to (a) use empty space in crate, (b) merge into Kasli, (c) make more powerful to sensibly occupy a slot, (d) become stand-alone
  • HVAMP32/8 would need to be (a) an RTM, (b) external, or (c) consolidated with their driver, or (d) CPCIs/DIOT/FMC style mezzanine
  • IDC-SMA/BNC/MCX would need to become RTMs or external VHDCI/HD68 boards (otherwise waste slots)
  • ad-hoc "mods" (e.g. Urukul/Mirny tunable VCO as PLLs) require more planning
  • provokes instinctive NIH+FUD reaction

The downsides drive and reinforce the need for:

  • External break-outs (due to connector density), see existing external breakout architectures: counter fixed slot positions/reduce slot waste
  • High-density digital IO from a single peripheral slot to replace the various 8x/16x DIO boards (see Banker, VHDCI): counter peripheral count decrease
  • Consolidation of mezzanines onto their carrier (consolidated hardware Stabilizer+Pounder Pounder#112, Almazny etc), or redesign mezzanine architecture (more FMC like), or into RTMs, or into CPCIs/DIOT-style mezzanines: to counter fixed slot positions/reduce slot waste
  • Well-defined cooling architecture/requirements
  • Well-defined identification/flashing/JTAG architecture
  • Well-defined powering/sequencing/health and status monitoring architecture

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions