Content for  TR 22.826  Word version:  17.2.0

Image guided surgery is gradually becoming mainstream. Currently, surgeons can plan a procedure based on the 3D display of patient anatomy reconstructed from MR (Magnetic Resonance) or CT (Computer Tomography) scans. And, in the near future, real-time medical imaging (3D ultrasound typically) can also be used as a live reference. Display devices such as e.g. head mounted display, monitors will be used to show real-time images merging the main video stream (endoscopy, overhead, microscopy…) with the live reference medical imaging. Metadata associated with the video can be updated at the frame rate (e.g., 3D position of probes). Only the method for conveying the multiple synchronized video/multi-frame sources along with their parameters (that may change at every frame) is specified in the present technical report. Mechanisms used for generating augmented reality views or to detect and to follow 3D position of devices are not addressed in this use case. One of the most challenging use cases for augmented reality assisted surgery is related to minimally invasive heart procedures where surgeons don't cut the breast bone but operate between the ribs. In fact, the heart is a moving organ with at rest, a cardiac cycle that lasts about 800ms (75 beats per minute) and is divided into two phases: diastole (about 60% of the cardiac cycle) and systole (40% of the cardiac cycle). Heart contraction itself involves a deformation of up to 25% of its total size (between 14cm and 16cm) and lasts around 200ms. This implies that the walls of the heart move at a speed of up to 20cm/s. In this use case, we examine a procedure called Coronary Artery Bypass Graft (CABG) which is a surgical procedure in which one or more blocked coronary arteries are bypassed by a blood vessel graft, usually taken from patient's arms or legs, to restore normal blood flow to the heart.

Sometimes, the left anterior descending (LAD) coronary is obstructed and requires coronary bypass surgery. However, it takes a deep intra-myocardial course and may be difficult to locate in the case of a thick epicardial adipose tissue or epicardial scar tissue. There are two basic ways of performing CABG: on-pump CABG and off pump CABG. In the on-pump case, the heart is temporarily stopped and blood flow is diverted using a heart-lung bypass machine. In the off-pump case, the area around the blocked coronary artery is stabilized using pods while the surgeon grafts the blood vessel on the pumping heart. Off-pump CABG is relatively a newer procedure to on-pump CABG and was shown to reduce postoperative complications associated with the use of cardiopulmonary bypass such as generalized systemic inflammatory response, cerebral dysfunction, myocardial depression, and hemodynamic instability. Stabilizing pods used during the procedure help to reduce the amplitude (and thereby the speed) of heart's movements at the place of the anastomosis. Thus, a safe assessment leads here to consider a relative speed (surgeon's hand and heart walls) of 3.5cm per second. Thus, limiting the error for the relative position of instruments and moving tissues to 0.5mm on the display monitors sets a maximum acceptable imaging system latency (including image generation, transmission, processing and display) of 14 ms. Considering 120fps frame rate and principles set forth in clause 5.2.1.3, the resulting cumulated end to end latency requirement falls around 1.5 ms. To guarantee correct recombination of the two data streams in a single and accurate A/R image by the A/R application, medical images are synchronised together thanks to local clocks stabilized on the grand master clock with a very tight accuracy. Due to computation challenges at the A/R application, it is expected that the scope will produce 4K uncompressed video, with the perspective to support also HDR (High Dynamic Range) for larger colour gamut management (up to 10 bits per channel) as well as HFR (High Frame Rate), i.e.; at least 120 fps to be able to track instruments and heart relative movements with enough precision. In addition, the data of a 3D ultra-sound probe is used to augment the main anatomical image with stiffness of soft tissues or Doppler information to the image. Various techniques exist to create a 3D ultra-sound image, ranging from combining 2D ultrasound slices to using dedicated 3D probes supporting 3D volume data in various coordinate systems and encodings. A 2D ultrasound typically produces a data stream of uncompressed images of 512x512 pixels with 32 bits per pixel at 20 fps (up to 60 fps in the fastest cases), resulting in a data rate of 160 Mbit/s up to 500 Mbit/s. Using 2D ultrasound to create a 3D ultrasound volume is typically slow and cumbersome. Dedicated 3D probes tend to work at higher data rates, i.e. above 1 Gbit/s of raw data, and are expected to reach multi gigabit data rates in future (e.g. producing 3D Cartesian volumes of 256 x 256 x 256 voxels each encoded with 24 bits at 10 volumes per second or better). Some processing such as lossless compression may be performed to reduce the data rates. Accurate recombination by the A/R application of images issued from both the laparoscope and the ultra-sound probe requires very precise location of the instruments inside the patient's body through a mechanism that is out of 5G system scope. However, location data shall be timestamped with same accuracy and made available at the A/R application at least at the same rate as the images. Typical CABG duration is two to four hours, this allows us to estimate targeted communication service availability figure for the successful transmission of images within latency constraints discussed above. In fact, considering that consecutive frame loss or delay may translate into a wrong estimated distance and may result in serious injury to the patient, we want this event to only happen with a very low probability during at least the duration of the procedure, a safe margin would be to consider ten hours of correctness in a row. Note that in this use case, a total of 240 images per second are exchanged over 5G communication service (120 images per second in each direction).

A patient is suffering from coronary artery disease and underwent a heart attack few hours ago. After being injected a clot-dissolving agent to restore blood flow in the blocked coronary artery he has been admitted to the hospital emergency room for a closed thorax coronary bypass surgery. The patient under anaesthesia is on the operating table and the surgery team is ready to start the procedure. Each needed equipment (laparoscope, ultra-sound probe, monitors …) is:

• Powered up,
• Subscribed to 5G-LAN type services deployed by the hospital IT infrastructure manager,
• Configured on which private groups they shall use to communicate with each other,
• Attached to the non-public 5G network covering the operating room.

In addition, the monitors are subscribed to a URLLC point-to-multipoint communication service dedicated to transport high data rate downlink video streams. The augmented reality application that handles the video streams generated by the laparoscope and the ultrasound probe is up and running and instantiated on private IT resources inside the hospital at a short network distance from the operating room.

 

1. The surgeon takes a blood vessel graft, usually from patient's arms or legs
2. Small incisions are performed in the patient's chest in order to insert a laparoscope equipped with a small video camera and a cold light source, and the ultra-sound mini-probe. Other small diameter instruments (grasper and scissors) and the cardiac stabilizer pods used for off-pump surgery are then introduced.
3. The application processes 4K images received from the laparoscope and the 3D representations issued by the ultrasound mini-probe to help the surgeon to determine the location of the left anterior descending (LAD) coronary and to visualize plaque and calcifications which may hamper coronary anastomosis suturing. Such processing requires both streams to be finely synchronized. Despite the low latency and high datarates, all medical imaging data is fully integrity protected as required by medical data protection regulations.
4. Since the heart is still beating during the procedure, all video streams have to be transferred to the application and from the application to the monitors with ultra-low latency to avoid perforating healthy heart tissue.
5. The surgeon performs the graft procedure by sewing one end of a section of the harvested blood vessel over a tiny opening made in the aorta and the other end over a tiny opening made in the blocked coronary vessel.

The procedure fully succeeded and the patient recovered in the hospital in a couple of days, including some time in the intensive care unit. The patient resumed his normal activities about three to four weeks after the bypass surgery and statistically show less post-operation complications than in classical procedure.

Jack is an obese patient with a Body Mass Index (BMI) above 40kg/m2 and suffers from End Stage Renal Disease (ESRD) and from diabetes and hypertension. As Higher BMIs in kidney transplant recipients are associated with excess risk of surgical site infections (SSIs), and negatively impact graft survival, Jack has been denied access to kidney transplant operation and need to undergo frequent and regular dialysis for now 5 years. The 5-year mortality rate for diabetic and hypertensive dialysis patient is unfortunately very high (around 70%) which makes Jack situation critical. The main hospital close to Jack's residence has develop a new, minimally invasive, robotic kidney transplantation method using a short midline epigastric incision that avoids any incision in the infection prone lower quadrants of the abdomen. A left kidney is procured from a 47-year-old Caucasian male, who died from a cerebrovascular accident. Subsequently, the hospital contacted Jack to propose him undergoing a RAKT. Now Jack, is under anaesthesia on the operating table and the surgery team is ready to start the procedure. Each needed equipment (robot, control console, monitors …) is:

• Powered up,
• Subscribed to 5G-LAN type services deployed by the hospital IT infrastructure manager,
• Configured on which private groups they shall use to communicate with each other,
• Attached to the non-public 5G network covering the operating room.

In addition, the monitors are subscribed to a URLLC point-to-multipoint communication service dedicated to transport high data rate downlink video streams. The application that handles the video streams generated by the laparoscope, runs the 3D patient body's model, sends control information to the robot, generates haptic feedback to the surgeon is up and running and instantiated on private IT resources inside the hospital at a short network distance from the operating room.

1. The surgeon positions at the console and the co-surgeon positions at the bedside and a 7cm periumbilical incision is made in order to insert a hand access device that will maintain peritoneal gas pressure while allowing for introduction of the graft.
2. Subsequently, four additional trocars are inserted for the right and left hand side robotic arms, the scope and the assistant.
3. After exposure of external iliac artery and vein, the right external iliac vein is clamped using plastic bulldogs, and the venotomy is made with the robotic Potts scissors at the site where the renal vein of the new kidney is to be sewn.
4. The graft is brought into the operative field through the midline incision and placed in the right lower quadrant. The renal vein attached to the new kidney is anastomosed in a continuous manner, end-to-side to the right external iliac vein.
5. Next, the right external iliac artery is clamped with plastic bulldogs. A circular arteriotomy is made and the renal artery is anastomosed end-to-side to the right external iliac artery.
6. After the anastomoses are tested and show no leak, the new kidney is re-vascularized (the plastic bulldogs are removed).
7. Last, the bladder is distended with saline and methylene blue and then dissected through the muscle layer in order to anastomose the ureter from the new kidney with the bladder.

During the operation, the surgeon at his console sends commands through his hands and feet movements to the robot over the 5G non-public network covering the operating room and receives both 3-dimensional images and haptic feedback from the robot. The critical medical application instantiated at network edge provides tactile guidance by constraining where the instruments (scalpel, etc.) can go. The surgeon is free to move inside the incisions, but each time he's going to the wrong place it feels like he's hitting a piece of glass or a constraint. The haptic feedback also allows to apply more consistent tension to suture material during robotic knot tying.

Thanks to the original robotic technique, the procedure fully succeeded and yielded low complications, rapid recovery, low pain and excellent graft function. The patient recovered in the hospital in 5 days, showing no graft rejection signs. He resumed a normal life without dialysis about three to four weeks after the transplant surgery and is submitted to regular follow-up exactly like patients with lower BMI.