{"id":7732,"date":"2023-09-25T20:20:35","date_gmt":"2023-09-25T19:20:35","guid":{"rendered":"https:\/\/dronesonen.usn.no\/?p=7732"},"modified":"2023-09-25T20:20:35","modified_gmt":"2023-09-25T19:20:35","slug":"turret-syndrome-week-5","status":"publish","type":"post","link":"https:\/\/dronesonen.usn.no\/?p=7732","title":{"rendered":"Turret Syndrome – Week 5"},"content":{"rendered":"\n

Here is our teams contributions for our project this week in sprint 5! Things are starting to ramp up!<\/p>\n\n\n\n

Harald Berzinis<\/h2>\n\n\n\n

Harald Berzinis<\/p>\n\n\n\n

This week I have been reiterating the original model, 3D-printing the trigger mechanism and further testing it.\u00a0\u00a0<\/p>\n\n\n\n

Initial CAD-design:<\/p>\n\n\n\n

<\/h2>\n\n\n\n

Main Enclosure:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The main enclosure that holds everything together, 3cm x 5cm x 11cm<\/p>\n\n\n\n

Bolt:
<\/p>\n\n\n\n

The main bolt that activates when the servo mechanism is initiated, converting from rotational to linear actuation for the trigger. This bolt is then suspended with a rubber band to the main enclosure. <\/p>\n\n\n\n

Servo bolt:
<\/p>\n\n\n\n

This servo bolt gets directly mounted on the 20 tooth spur gear and the stopping component for the main bolt, this of course gets placed through the main enclosure, then glued and assembled with the other parts.<\/p>\n\n\n\n

Stopping component for the main bolt:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

This component is responsible for holding the main bolt at the suspended state. When the servo is initiated, this component rotates to the left and releases the suspended bolt hitting the trigger.<\/p>\n\n\n\n

Trigger bolt<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

This is an extension for the main bolt so it hits the trigger.<\/p>\n\n\n\n

20 and 10 tooth gears:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

These are the 20 and 10 tooth spur gears, which makes a ratio of 2:1. These gears were reiterated due to when 3D-printing I experienced that the gears that I made were not high enough resolution, therefore I stepped down the number of teeth on each gear. These were printed at 70% infill to make sure that they do not break under load.<\/p>\n\n\n\n

Assembly and testing:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Fully assembled trigger mechanism. In addition, we have tested it and the mechanism activates the paintball gun’s trigger. Further testing proved that the mechanism experiences tear at the main bolt and that it needs to be printed at a higher level of infill.<\/p>\n\n\n\n

Next week:<\/p>\n\n\n\n

The plan for me next week is to make a version of this that is mounted on the buttstock of the paintball gun, this is because we have a lot of unused space. I will then reiterate this mechanism to a more beefier version to withstand the rubber band force and further improve the stability and safety for the shooting mechanism.<\/p>\n\n\n\n

Mats Bergum<\/h2>\n\n\n\n

Most of my time was spent watching and following this tutorial. (https:\/\/www.youtube.com\/watch?v=d_TPIxPX01s&ab_channel=RobertFeranec<\/a>) from Robert Feranec. Following this tutorial taught me how to make footprints and 3D models and assign them to the correct components. I also learned how to set up PCB stackup in Allegro and route the PCB in Allegro. Moreover, the tutorial also showed me how to import changes in the schematic to the existing PCB design, which can be helpful when producing my own PCB.<\/p>\n\n\n\n

The PCB I made following this tutorial can be found below. This was done by copying what was done in the tutorial, but it was a good learning experience.<\/p>\n\n\n\n

3D view of the board:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

Routing:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

I tried to make the PCB design for our circuit, but I could not do this due to some console errors that I could not fix. Thus, I only made some footprints and 3D models for curtain components. Next week, I need to try fixing those errors and start with the routing.<\/p>\n\n\n\n

Christopher Daffinrud<\/h2>\n\n\n\n

After severe testing using the Pi for object detection calculations, and using probably one of the most efficient models in EfficientDet we concluded in the beginning of week 5 that we would like to separate the object and color recognition calculations on an external device communicating with the Raspberry Pi over UDP. <\/p>\n\n\n\n

Our new intention is setting up a local network on the Raspberry Pi allowing for UDP-communication between the Pi and an external computer with a socket. The camera feed captures from the Raspberry Pi\u00b4s web camera shall then be transferred to the external computer for computations. The external computer will then return the relevant data and coordinates needed for the Turret System to perform its actions depending on the calculations.<\/p>\n\n\n\n

I iterated on my previous class diagram, now facilitating this change.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

As you can see from the model the Turret and the external computer are connected through the class \u201ccommInterface\u201d.<\/p>\n\n\n\n

    \n
  • UDPSocket for transferring camera frames from Pi to external computer<\/li>\n\n\n\n
  • TCPSocket for transferring object coordinates and relevant data from external computer to Pi<\/li>\n<\/ul>\n\n\n\n

    I also created a simple high-level component diagram of our system, including the component needing 5V or 12V.<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    The 3.3V to 5V represents the converter circuit which Harald and Mats are developing.<\/p>\n\n\n\n

    All nodes with 5V and 12V are additional voltage requirements needed that will be fulfilled by creating a PCB.<\/p>\n\n\n\n

    The network communication channel in this model is the Local Wifi-network supplied either by the Raspberry Pi or the external computer with communication through sockets. <\/p>\n\n\n\n

    These diagrams allowed further specification of tasks needed to be done through Azure Devops. <\/p>\n\n\n\n

    Hopefully we will be able to start implementing some software solutions during this week. That will also be the main focus for software next week.<\/p>\n\n\n\n

    Ole Eirik S.Seljordslia<\/h2>\n\n\n\n

    In response to the computational limitations we encountered with the Raspberry Pi last week, we took a step back to explore alternative solutions. One of the promising ideas that emerged was the concept of streaming video from the Raspberry Pi to an external computer. This approach would leverage the secondary computer’s superior processing power to handle computationally intensive tasks.<\/p>\n\n\n\n

    In this envisioned setup, the external computer would receive the video feed from the Raspberry Pi, process it, and return coordinates corresponding to any detected balloons within the video stream. An advantage of this arrangement is its versatility; it allows us to conveniently test the system using a regular laptop, we could also switch out the laptop at a later time.<\/p>\n\n\n\n

    To realize communication between the turret and the image processor, we needed a interface capable of efficiently sending and receiving frames. In practice, the external computer would act as a server hosting a socket connection through UDP, enabling the Raspberry Pi mounted on the turret to send images and receive balloon coordinates in response. These coordinates represent the predictions made by our object detection model regarding balloon locations.<\/p>\n\n\n\n

    During the past week, I dedicated my efforts to implementing and testing this interface between the two computers.<\/p>\n\n\n\n

    Our project consists of two distinct code bases: one for the turret and another for the image processor. Given that the turret’s role is to transmit frames to the image processor, I developed communication classes to interface the two systems. The following snippet is the turret’s method for sending frames: <\/p>\n\n\n\n

    Transmitting frames<\/h3>\n\n\n\n
     1\n 2\n 3\n 4\n 5\n 6\n 7\n 8\n 9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20\n21\n22\n23\n24\n25\n26<\/pre><\/td>
    def<\/span> send_frame<\/span>(<\/span>self<\/span>,<\/span> frame:<\/span> cv2.typing.MatLike)-><\/span>None<\/span>:<\/span>\n        """_summary_<\/span>\n        Send individual frames<\/span>\n        Args:<\/span>\n            frame (cv2.typing.MatLike): Frame from video stream<\/span>\n        Raises:<\/span>\n            Exception: Unable to encode frame<\/span>\n        """<\/span>\n        return_value,<\/span> image_buffer<\/span> =<\/span> cv2.imencode(<\/span>self<\/span>.__image_format,<\/span> frame)<\/span>\n        if<\/span> not<\/span> return_value:<\/span> raise<\/span> Exception<\/span>(<\/span>'Unable to encode frame'<\/span>)<\/span>\n        image_buffer<\/span> =<\/span> image_buffer.tobytes()<\/span>\n        number_of_packets<\/span> =<\/span> 1<\/span>\n        if<\/span> len<\/span>(image_buffer)<\/span> ><\/span> self<\/span>.__max_buffer_size:<\/span>\n            number_of_packets<\/span> =<\/span> int<\/span>(np.ceil(<\/span>len<\/span>(image_buffer)\/<\/span>self<\/span>.__max_buffer_size))<\/span>\n       \n        self<\/span>.__socket.sendto(pickle.dumps({<\/span>'packets'<\/span>:number_of_packets}),<\/span>\n                            self<\/span>.__server)<\/span>\n        left<\/span> =<\/span> 0<\/span>\n        right<\/span> =<\/span> self<\/span>.__max_buffer_size<\/span>\n        for<\/span> packet_index<\/span> in<\/span> range<\/span>(number_of_packets):<\/span>\n            payload<\/span> =<\/span> image_buffer[left:right]<\/span>\n            left<\/span> =<\/span> right<\/span>\n            right<\/span> +=<\/span> self<\/span>.__max_buffer_size<\/span>\n            self<\/span>.__socket.sendto(payload,<\/span> self<\/span>.__server)<\/span>  \n<\/pre><\/td><\/tr><\/table><\/div>\n\n\n\n
    This ‘send_frame’ method anticipates a ‘cv2.typing.MatLike’ input, which aligns with the output of ‘cv2.VideoCapture()’. It encodes the frame to a buffer using the ‘.jpg’ format. If the encoding is successful, it converts the buffer to bytes and determines the number of packets required based on the byte conversion. The number of packets is then sent as the first packet to the image processor, serving as an indicator of the frame’s completeness. This packet is serialized through the `pickle.dumps` function. Subsequently, segments of the image buffer are dispatched to the image processor, respecting the maximum packet size, which should be predefined and agreed upon by both the turret and image processor. The maximum packet size is determined by each computer UDP MTU, this can vary based on hardware and operating system compositions. <\/p>\n\n\n\n
    The image processor, in turn, implements a complementary method to receive frames:<\/p>\n\n\n\n
    Receiving frames<\/h3>\n\n\n\n
     1\n 2\n 3\n 4\n 5\n 6\n 7\n 8\n 9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20\n21\n22\n23\n24\n25\n26\n27\n28\n29\n30\n31\n32\n33\n34\n35\n36\n37\n38\n39\n40<\/pre><\/td>
    def<\/span> receive_frame<\/span>(<\/span>self<\/span>)->np.ndarray:<\/span>\n        """Will return next frame from a video stream<\/span>\n        Returns:<\/span>\n            Array of pixels<\/span>\n        """<\/span>\n        buffer<\/span>,<\/span> address<\/span> =<\/span> self<\/span>.__socket.recvfrom(<\/span>self<\/span>.__max_buffer_size)<\/span>\n        # try:<\/span>\n        #     frame_info = pickle.loads(buffer)<\/span>\n        # except:<\/span>\n        #     print('Missed frame information')<\/span>\n        #     return None<\/span>\n        if<\/span> len<\/span>(<\/span>buffer<\/span>)<\/span> ><\/span> 500<\/span>:<\/span> return<\/span> None<\/span>\n        frame_info<\/span> =<\/span> pickle.loads(<\/span>buffer<\/span>)<\/span>\n        if<\/span> frame_info:<\/span>\n            number_of_packets<\/span> =<\/span> frame_info[<\/span>'packets'<\/span>]<\/span>\n            packets<\/span> =<\/span> bytes<\/span>()<\/span>\n           \n            for<\/span> packet_index<\/span> in<\/span> range<\/span>(number_of_packets):<\/span>\n                payload,<\/span> _<\/span> =<\/span> self<\/span>.__socket.recvfrom(<\/span>self<\/span>.__max_buffer_size)<\/span>\n                packets<\/span> +=<\/span> payload<\/span>\n            return<\/span> self<\/span>.__process_frames(packets)<\/span>\n       \n    def<\/span> __process_frames<\/span>(<\/span>self<\/span>,<\/span> buffer<\/span>)->np.ndarray:<\/span>\n        """Transform frame.<\/span>\n        Args:<\/span>\n            buffer: Incoming buffer representing a frame from video stream.<\/span>\n        Returns:<\/span>\n            Transformed buffer of frame<\/span>\n        """<\/span>\n        frame<\/span> =<\/span> np.frombuffer(<\/span>buffer<\/span>,<\/span> dtype=np.uint8)<\/span>\n        frame<\/span> =<\/span> frame.reshape(frame.shape[<\/span>0<\/span>],<\/span> 1<\/span>)<\/span>\n        frame<\/span> =<\/span> cv2.imdecode(frame,<\/span> cv2.IMREAD_COLOR)<\/span>\n        frame<\/span> =<\/span> cv2.flip(frame,<\/span> 1<\/span>)<\/span>\n        return<\/span> frame<\/span>\n<\/pre><\/td><\/tr><\/table><\/div>\n\n\n\n
    The ‘receive_frame’ method begins by receiving a buffer from the turret. It extracts frame information from the first packet, particularly the number of packets to expect. To address potential challenges like missing initial packets, we employ a size threshold to distinguish between the initial information packet and subsequent data packets. This could have also been handled by trying to process the packet and possibly catching an exception. I opted from this as I expect it to be a more time consuming solution opposed to checking the length.  <\/p>\n\n\n\n
    Following this, each packet is collected into a byte array before being processed by the internal `__process_frames` method, which ultimately returns a NumPy array suitable for object detection.<\/p>\n\n\n\n
    I created two examples to put these methods to the test.<\/p>\n\n\n\n
    Image processor example<\/h3>\n\n\n\n
     1\n 2\n 3\n 4\n 5\n 6\n 7\n 8\n 9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20\n21\n22\n23\n24\n25\n26\n27\n28\n29<\/pre><\/td>
    from<\/span> image_processor.communication<\/span> import<\/span> Communication<\/span>\nimport<\/span> cv2<\/span>\nimport<\/span> numpy<\/span> as<\/span> np<\/span>\nimport<\/span> time<\/span>\ncommunication<\/span> =<\/span> Communication(<\/span>"192.168.1.77"<\/span>)<\/span>\nprevious_frame<\/span> =<\/span> 0<\/span>\nnext_frame<\/span> =<\/span> 0<\/span>\ni<\/span> =<\/span> 0<\/span>\nfps_sum<\/span> =<\/span> 0<\/span>\nfps_str<\/span> =<\/span> 'loading'<\/span>\nwhile<\/span> True<\/span>:<\/span>\n    frame<\/span> =<\/span> communication.receive_frame()<\/span>\n    if<\/span> frame<\/span> is<\/span> not<\/span> None<\/span> and<\/span> type<\/span>(frame)<\/span> ==<\/span> np.ndarray:<\/span>\n        next_frame<\/span> =<\/span> time.time()<\/span>\n        fps<\/span> =<\/span> 1<\/span>\/(next_frame<\/span> -<\/span> previous_frame)<\/span>\n        previous_frame<\/span> =<\/span> next_frame<\/span>\n        fps_sum<\/span> +=<\/span> fps<\/span>\n        i+=<\/span>1<\/span>\n        if<\/span> i<\/span> ==<\/span> 10<\/span>:<\/span>\n            fps_str<\/span> =<\/span> str<\/span>(<\/span>int<\/span>(fps_sum\/i))<\/span>\n            fps_sum<\/span> =<\/span> 0<\/span>\n            i<\/span> =<\/span> 0<\/span>\n        cv2.putText(frame,<\/span> fps_str,<\/span> (<\/span>7<\/span>,<\/span> 70<\/span>),<\/span> cv2.FONT_HERSHEY_SIMPLEX,<\/span> 3<\/span>,<\/span> (<\/span>100<\/span>,<\/span> 255<\/span>,<\/span> 0<\/span>),<\/span> 3<\/span>,<\/span> cv2.LINE_AA)<\/span>\n        cv2.imshow(<\/span>"Stream"<\/span>,<\/span> frame)<\/span>\n       \n        if<\/span> cv2.waitKey(<\/span>1<\/span>)<\/span> ==<\/span> 27<\/span>:<\/span>\n            break<\/span>\n<\/pre><\/td><\/tr><\/table><\/div>\n\n\n\n
    Turret example<\/h3>\n\n\n\n
     1\n 2\n 3\n 4\n 5\n 6\n 7\n 8\n 9\n10\n11\n12\n13\n14<\/pre><\/td>
    from<\/span> turret.communication<\/span> import<\/span> Communication<\/span>\nimport<\/span> cv2<\/span>\nci<\/span> =<\/span> Communication(<\/span>'192.168.1.77'<\/span>)<\/span>\ncamera<\/span> =<\/span> cv2.VideoCapture(<\/span>0<\/span>)<\/span>\nwhile<\/span> True<\/span>:<\/span>\n    value,<\/span> frame<\/span> =<\/span> camera.read()<\/span>\n    if<\/span> value:<\/span>\n        ci.send_frame(frame)<\/span>\n<\/pre><\/td><\/tr><\/table><\/div>\n\n\n\n
     The turret example sends frames to the image processor, while the image processor example displays the frames and calculates an average frames-per-second (FPS) metric. During the testing, I achieved an average of approximately 30 FPS, this was quite satisfactory as the camera used by the turret only outputted 30 FPS.<\/p>\n\n\n\n
    Hannes Weigel<\/h2>\n\n\n\n
    Week 5<\/h4>\n\n\n\n
    The goal for this week was to assemble the Azimuth Drive of the turret. Most of the printing work had been achieved by late week 4 \/ early week 5. With an estimated total printing time of 70 hours, I\u2019m certain this project has contributed to about 0.1% of the total microplastics in the oceans.<\/p>\n\n\n\n
    The Sun Gear<\/h4>\n\n\n\n
    <\/p>\n\n\n\n
    The sun gear is now completely finished. Consisting of the outer teeth ring, upper inner race, lower inner race, and 144 bearing rollers.<\/p>\n\n\n\n
    <\/p>\n\n\n\n
    The inner race of the slew bearing consists of an upper and lower ring. The seam of these rings is offset by 45 degrees. <\/p>\n\n\n\n
    After assembling both upper and lower, I realized that I had no way of feeding the bearings into the race.<\/p>\n\n\n\n
    <\/p>\n\n\n\n
    Therefore I had to bolt the upper plate of the thrust bearing to the lower ring of the inner race.<\/p>\n\n\n\n
    With these two components now joined, the painstaking process of adding 144 bearings to the race started. Thankfully, I got great help from Ole, as the entire process of cleaning the bearings, and orienting the bearings took nearly an hour.<\/p>\n\n\n\n
    <\/p>\n\n\n\n