Finding the right integrations, libraries, and open source tooling to extend a product has long been a challenge for developers. While Datadog has a vast offering of monitoring and observability solutions, many teams need to customize their setup in some way-whether by extending the Datadog Agent, integrating with third-party services, or using SDKs to interact with the Datadog API. Previously, these resources were spread across various repositories and documentation pages, which made it challenging for developers to find the most suitable tool for their needs.

We are pleased to announce the launch of the Datadog Developer Hub, a centralized resource for developers to find components that enable them to extend the capabilities of Datadog. The Developer Hub provides a searchable catalog of official Datadog and community-contributed integrations and libraries, learning resources on how to build integrations, and information on how to join the community of Developer Hub contributors.

Access a collection of tools to extend Datadog

The Developer Hub brings together a variety of tools that developers can use in conjunction with Datadog, including:

• Agent integrations: Agent integrations enable you to extend the Datadog Agent by specifying the data it can ingest, how it processes that data, and where it sends it.
• Core Datadog integrations: Core integrations are maintained by Datadog and provide robust, out-of-the-box functionality for monitoring databases, cloud providers, messaging systems, and more.
• Libraries: These include Datadog and community-contributed API and DogStatsD client libraries, APM and Continuous Profiler libraries, serverless client libraries, command-line tools, and UI wrappers around the Datadog API.
• Sample apps: A growing collection of sample apps enable you to kick-start your Datadog instrumentation journey.

Check out the Datadog Developer Hub today, your go-to destination for discovering integrations, libraries, tools, and learning resources that extend Datadog’s capabilities. The Developer Hub will continue to grow as new resources are added to help you customize and enhance your observability workflows. If you aren’t yet a Datadog user, you can sign up for a free 14-day trial.

Originally published at https://www.datadoghq.com on May 20, 2025.

Datadog Ambassadors share their expertise through blog posts, conference talks, open source contributions, and community leadership, helping developers around the world understand observability, DevOps, security, and more. From leading community events to building custom integrations, our Ambassadors have been hard at work showcasing their Datadog chops.

What we’ve been up to

Over the past year, our Ambassador community has continued to grow-not only in size, but also in impact. From publishing technical blog posts to giving conference talks and growing local communities, our Ambassadors have been at the forefront of thought leadership in the observability space.

Whether it’s community organizing or sharing real-world experiences on cloud migrations and platform engineering, our Ambassadors continue to produce thoughtful, hands-on content. Some standout contributions from this year include:

Several Ambassadors have taken on leadership roles within their local communities, scaling User Groups into thriving hubs of learning and collaboration. From meetups to workshops, they’re helping foster strong regional ecosystems that support ongoing learning and connection.

• Ichiro Kano has played a key role in growing the Datadog Japan User Group (JDDUG) into a vibrant community, regularly organizing in-person meetups and technical sessions that bring together engineers across Japan to share observability best practices.
• Changhyeon Yoon has been instrumental in growing the Datadog Korea User Group (DDKruG), helping organize meetups that highlight real-world use cases, foster community-driven learning, and build a stronger network of Datadog users across Korea.

This year, the Ambassador program is expanding with a new cohort of builders, security experts, educators, and community leaders-and we’re excited to welcome them to the community.

Meet the 2025 Datadog Ambassadors

Kelly is a Staff Security Engineer at Stavvy, focused on building practical, scalable security programs. His expertise spans detection engineering, cloud security, SIEM, and security automation-driving initiatives like detection as code, incident response, and compliance enablement. At DASH 2024, he shared a “crawl, walk, run” framework for detection as code, helping teams bring structure and scale to their detection efforts. A passionate problem solver and active member of the Datadog Slack community, Kelly brings high energy and curiosity to everything he does-whether it’s architecting security solutions or diving into technical challenges in his home lab.

Rebecca is a Cybersecurity Engineer at AssessFirst, where she leads efforts in governance, SOC, Purple Team operations, and ISO 27001 compliance. She’s particularly committed to cybersecurity governance and awareness-raising, with the passion of making this complex field accessible to everyone on her Medium blog. As a member of CESIN, she’s also passionate about sharing knowledge and disseminating best practices, contributing to the development of the cybersecurity community in France.

Shogo Hasunuma is based in Japan and serves as the head of the Managed Service Provider (MSP) Section at iret, Inc. He joined the company in 2015 with no prior experience in IT engineering and has steadily built his career-from an entry-level monitoring operator to roles in infrastructure design and implementation-before being appointed as a team leader in 2019. Currently, he is committed to enhancing managed services by advancing incident response automation, adopting generative AI, and implementing comprehensive observability strategies centered around Datadog. He also supports end users in fostering autonomous DevOps practices by offering guidance on effective usage of Datadog.

YoungJin is a DevOps Engineer driving infrastructure modernization at LG UPlus. He specializes in solving complex challenges that emerge in enterprise-scale environments, focusing on building scalable, resilient systems through modern DevOps practices. With a strong emphasis on observability and performance tuning, he utilizes Datadog to instrument services, analyze performance metrics, and proactively improve system reliability and the overall user experience. He is an active AWS Community Builder, contributing to the growth of cloud-native practices within the community, and also serves as a HashiCorp Ambassador, advocating for infrastructure as code and automation across large-scale deployments. You can read more about his DevOps adventures on his blog.

Yubin is a site reliability engineer at Karrot, focused on building resilient, scalable systems that internal developers can trust. She’s passionate about improving reliability through comprehensive monitoring, thoughtful CI/CD design, and pragmatic incident management. Yubin is a dedicated community member, always eager to share and learn alongside fellow engineers. She regularly publishes a number of blog posts on her personal blog on Datadog, Kubernetes, and observability events in Korea.

Michael is a Kubernetes and platform engineering expert, author, consultant, and CNCF Ambassador. With a knack for turning complexity into clarity, he helps companies around the world level up their infrastructure-and regularly shares his expertise through blogs, books, courses, podcasts, and conference talks.

Jon is an Engineering Manager at Axis Communications, overseeing the core services team for Axis Cloud Connect. With a background in architecture and DevOps, he brings a strategic and technical lens to platform engineering. Jon has shared his insights at events like DASH 2024 and AWS Summits, covering topics like platform engineering, cloud transformation, and DevOps practices in real-world settings.

Logan is a cloud and observability lead at RapDev, a Datadog Premier Partner. With experience spanning cloud automation, network engineering, and system administration, Logan plays a key role in helping organizations implement and optimize their observability strategies in the most automated and scalable fashion possible. He’s deployed nearly every Datadog product and written more than a dozen custom integrations. Logan also contributed to an open source utility for managing secrets with the Datadog Agent, which was donated to Datadog in 2024. Check out more of his work on RapDev’s blog.

Nilton is a Senior Data Executive in Business Intelligence, Data Engineering, Machine Learning and Generative AI at Deloitte, with over a decade of experience in multinational and global companies, leading large-scale corporate strategic initiatives. A longtime advocate of knowledge sharing, he actively contributes to the Brazilian tech community through technical talks, blog posts, and open discussions on cloud-native best practices.

You can check out the full list of Datadog Ambassadors and read more about the program here.

Thank you, Ambassadors!

The Datadog Ambassadors program continues to grow as a vibrant community of technical leaders, content creators, and practitioners who are shaping how teams around the world think about observability, reliability, and performance. We’re excited to see how this year’s Ambassadors will continue to inspire others in the year ahead.

Want to be an Ambassador? Learn more about the program here.

Originally published at https://www.datadoghq.com on May 13, 2025.

My outdoor garden, in particular, has always been focused on flowers. I’ve carefully tended to a mix of annuals and perennials throughout the seasons, killing and bringing to life many species of flora over the years, trying to put the jungle in my concrete jungle apartment.

I’ve always been intrigued by the idea of growing my own vegetables and herbs. The thought of cultivating fresh produce is appealing, but I’ve often felt overwhelmed by the perceived complexity-the care, knowledge, monitoring, and maintenance that produce requires.

I began to wonder: could I create a system that allows me to grow plants with the precision and care of a professional botanist and transform from city girl to farmer with a few lines of code?

This curiosity led me to the concept of building a hydroponic garden indoors.

My vision is to grow herbs and vegetables from the comfort of my own home while integrating hardware and software to collect data, leverage machine learning, and manage my garden remotely. This system would provide real-time insights into my garden’s needs, ensuring each plant gets exactly what it needs to thrive.

Developing my hydroponic gardening system

My research began with exploring existing indoor hydroponic grow systems.

It felt like a real-life Goldilocks situation: one system was far too extravagant-both in size and cost, priced at several months’ rent-yet it featured an appealing remote monitoring app. Another system was too small, producing only three herb varieties per month, which wouldn’t satisfy my constant need for cilantro.

As I dug deeper into the soil of my research, I stumbled upon online communities that built their own hydroponic setups. I was surprised to discover just how vast this sub-community is, with enthusiasts creating everything from compact desktop systems to large-scale setups that fill entire backyards, growing enough arugula to feed entire neighborhoods.

This vibrant community inspired me to build my own setup with Viam- one where I could have complete control over my plants’ growth, lighting, water, and nutrient distribution, and, most importantly, design the smart monitoring system of my dreams.

Leaning into the Kratky method for hydroponics

What is the Kratky method?

After researching various hydroponic techniques, I used the Kratky method for my garden. The Kratky method is a passive form of hydroponics, making it ideal for beginners or anyone looking for a simple, cost-effective setup.

This approach allows me to monitor the root system and water nutrient levels closely, which is crucial for ensuring healthy plant growth.

How does the Kratky method work?

The Kratky method is essentially a simplified version of the Deep Water Culture (DWC) technique, another common hydroponic method. In a typical DWC setup, plants are suspended in special pots or nets, with their roots fully submerged in a reservoir of aerated, nutrient-rich water.

This setup provides continuous access to nutrients while requiring air pumps to oxygenate the water.

The Kratky method, however, offers a clever twist on the DWC approach by eliminating the need for air pumps. Instead, it maintains a 3–4 cm gap between the plant holder and the water surface. This gap allows air to circulate around the roots, providing the necessary oxygen without the need for additional equipment.

By suspending the plants slightly above the water level instead of having them float directly on top, the Kratky method simplifies the overall hydroponic setup. It reduces both the cost and complexity while still providing an efficient way to grow a variety of plants indoors.

Designing my hydroponic garden

Hydroponics is relatively straightforward: germinate seeds, grow them in oxygenated water without soil, and, ideally, enjoy the benefits of a flourishing indoor garden.

To keep costs low, I started by exploring inexpensive PVC pipe systems that I could build indoors.

‍Setting up visual tracking

To document the growth process of my plants, I wanted to create a timelapse with images and use the images to train custom ML models. For this, I’m using Viam to collect image data from a camera, capturing every stage of my plants’ development.

Monitoring the garden’s environment and health

Alongside visual tracking, I needed to to monitor crucial environmental factors such as ambient temperature, humidity, and other conditions to ensure my plants were thriving.

Viam’s built-in sensor modules make it easy to monitor these variables without having to write extensive code, streamlining the setup and management of the system.

Maintaining the right nutrient balance is essential in hydroponics, so I needed a way to track the pH levels of the water and the nutrients I’m adding. I found an affordable pH sensor compatible with a Raspberry Pi, which I plan to integrate into Viam’s Registry by creating a custom module.

I’m also incorporating flow sensors to monitor water circulation and ensure that plant roots aren’t clogging the system.

By gathering all this data, I can create a customized app using Viam, centralizing all the monitoring and data collection. This will allow me to build a truly smart hydroponic system with all the functionalities I need to maintain my garden effortlessly.

Kicking off germination

To kickstart my indoor hydroponic garden, I bought a variety of seeds to experiment with and chose those with relatively low germination times. I decided to begin with arugula, basil, buttercrunch lettuce, cilantro, parsley, and bibb lettuce-all favorites that I love to use in salads and as meal toppings.

The germination process itself is quite hands-on: I place the seeds between damp paper towels, moisten them with filtered water to maintain a neutral pH, and then seal them in Ziploc bags to create a humid environment.

Then comes the most exciting part-waiting for the first signs of sprouting and building a robot to monitor that process.

Building the Germination Station

While waiting for the seeds to sprout, I used Viam to set up a simple yet effective monitoring system to ensure ideal conditions for germination. I built this system using a webcam, a BME680 environmental sensor, and a Raspberry Pi, allowing me to capture thousands of images to visually track the growth process.

Alongside the images, I also gathered sensor data to monitor critical environmental factors such as temperature, pressure, humidity, and gas levels-all essential for maintaining a balanced greenhouse environment.

For successful germination, I aimed to keep the temperature within the ideal range of 65–70 degrees Fahrenheit. Initially, I placed the germination bags by a window in a plastic container, but the sensors quickly showed that temperatures reached up to 90 degrees in direct sunlight.

This data alerted me to relocate my seedlings promptly to prevent them from overheating and ensure they stayed in a nurturing environment.

Taking my DIY hydroponic system to the next level

This project began with a simple question: Can I grow my herbs and vegetables indoors with the precision of a professional botanist?

By combining a DIY hydroponic setup with a smart monitoring system using Viam, I was able to create an environment where I could control and optimize every aspect of plant growth, from the nutrient levels in the water to environmental conditions.

Using sensors, a webcam, and a Raspberry Pi 5, I’ve built a system to help monitor key variables like temperature, humidity, and the visual rate of growth remotely to ensure ideal growing conditions. This approach has not only demystified the process of hydroponic gardening but also made it more accessible and manageable for a small indoor setting.

There’s still work to be done, many variables to test and refine, and more data to collect as the plants grow. The progress so far proves that with the right tools and the power of open-source software, it’s entirely possible to design a smart hydroponic system. The next steps in this project include:

• Training custom ML models on the plants’ growth to use that data for future crops
• Building a module for a pH sensor to track nutrient changes in the hydroponic setup
• Building a custom monitoring app leveraging Viam’s APIs to have a dashboard of information available about the garden

Build your own hydroponic gardening system

If you’re considering starting your own smart hydroponic garden, don’t let the perceived complexity hold you back- start small, learn as you go, build your system modularly, and build a system with Viam that fits your plant-growing needs.

Stay tuned for my step-by-step tutorial to help you get started. In the meantime, check out our Codelabs for more tutorials, and if you have any questions, join the conversation in our Discord community or connect with me on TikTok!

Originally published at https://www.viam.com.

Computer vision enables robots to perceive and interact with their surroundings. Viam’s Vision Service supports advanced capabilities such as real-time object detection, classification, and 3D segmentation, allowing robots to understand and dynamically respond to the world. Additionally, Viam offers custom model training using images collected with Viam, allowing developers to tailor solutions to specific applications. But with the vast array of datasets and models available with Hugging Face, how can developers leverage these resources to enhance robotic capabilities even further?

To harness the power of Hugging Face models on Viam machines, the Viam community has contributed custom Vision Service modules that integrate YOLOv5 and YOLOv8 inference libraries to the registry for real-time detections. Configuration and testing of these models on hardware requires no code up front. The YOLOv5 module offers ease of use, making this model a popular choice for developers looking to quickly deploy computer vision models. The YOLOv8 module, on the other hand, is designed for speed and accuracy, making it ideal for applications that require real-time object detection.

The choice between the two depends on the specific requirements of the application, whether it’s ease of deployment or the need for high-performance models in dynamic environments. There are other modules on the Viam registry that leverage other Huggingface models like LLMs and beyond, making it easy for developers to leverage AI models from Hugging Face on their machines in the real world.

Deploying a Hugging Face model on a Viam machine

To use this module, first create a machine instance on the Viam app. For this guide, the only hardware necessary will be a computer to run viam-server and a webcam to show detections or classifications from a Hugging Face model of choice.

Set up the machine according to the instructions in the Set up your machine part guide in the Viam app. Once the machine is connected and live, head to the Configure tab to begin configuring a webcam and the vision service that will run the Hugging Face model.

In the Builder panel, add a component using the ‘+’ icon. Select ‘Component’. Search for the camera model ‘webcam’ and add it to the machine configuration.

Next, add a vision service that will leverage the configured camera to run detections. Using the ‘+’ icon, select ‘Service’ and search for the ‘vision / yolov8’ model to add to the machine configuration. Either YOLO module will work for this tutorial.

In order to leverage a Hugging Face model with a Viam YOLO module, find a model compatible with the model inference library selected. Browse through the Hugging Face model database and select a compatible model. In the GitHub example, a model for hard hat detection is used, which can be used on a security system designed for construction sites to ensure safety measures are followed.

Being a sneaker lover and having a closet that is overflowing, I want to use a shoe classification model to help me figure out what percentage of shoe brands I have in my sneaker collection to help me downsize (or have an excuse to buy more). I’m selecting YOLOv8 Shoe Classification Model from the Hugging Face models library, uploaded by user @keremberke.

After the module is added to the machine, configure the custom attributes for the Vision Service. The following attributes are available for the model:

Name: model_location

Type: string

Inclusion: Required

Description: Local path or HuggingFace model identifier

Because I am using a model hosted on Hugging Face, all I need to add is the path after the URL slug.

{
"model_location": "keremberke/yolov8n-shoe-classification"
}

If using a locally downloaded model, the configuration syntax is as follows:

{
"model_location": "/path/to/yolov8n.pt"
}

The final configuration step is to configure a transform camera component, which is a pipeline for applying transformations to an input image source. Add a new Component, search for ‘camera / transform’, and configure the pipeline as shown. For this transform camera, the transformations will be layered over the webcam feed to show classifications in real-time. Specify the classifier name, which will be the previously configured vision service named ‘yolov8’. Add the ‘webcam’ in the ‘Depends On’ attribute field. If a detector is chosen, follow the instructions to set up detections for a transform camera.

Once the pipeline is set up, it is time to test the Hugging Face model on a webcam, showing real-time classifications. This setup works whether you are testing on a computer or have configured a machine that exists out in the world, allowing AI capabilities to enhance the performance of a machine in many applications.

Next steps

By leveraging the YOLO modules in Viam’s Registry, developers can use state-of-the-art object detection algorithms modularly with minimal upfront coding. This flexibility allows for rapid prototyping and deployment, catering to a wide range of applications from security systems to automation in a warehouse to cleaning up your closet for your next home improvement project. After testing Hugging Face models on a Viam machine, the next step is to write custom code using Viam’s unified API’s and flexible SDK’s offered in different languages. Explore the Viam Registry and contribute to the open-source ecosystem by adding more features, models, and integrations for machine learning applications. Whether organizing your closet or creating a new home security system: Computer vision has so many potential applications, go start building your next prototype with Viam today.

Deploying Hugging Face models on any robot in the real world was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

Enter Tableau, a popular and powerful tool that’s a game-changer for data visualization. By leveraging Tableau, you can transform heaps of raw sensor data from Viam’s data management service into insightful visualizations. This not only makes the data easier to interpret but also enhances inference, decision-making, trend predictions, and more.

Connecting Viam sensor data to Tableau

The first step in making beautiful charts and beyond with sensor data through Viam is to collect enough data to visualize. You can learn how to do this in our Data Capture docs. Make sure to enable cloud sync so you can access all of your data from the data management service where the sensor data is synced and stored.

The next step is to configure database access for your Organization by using the Viam CLI to set up a new read-only database user for the Viam organization’s data. Using this direct database connection in Tableau enables seamless data importing for your visualization.

You will need to use Tableau for Desktop, instead of the cloud version, to configure the direct database connectivity with the MongoDB Tableau Connector.

First, download the Mongo DB JDBC Driver and move the downloaded .jar file into the appropriate directory for your operating system. Then, download the MongoDB Tableau Connector and move the downloaded .taco file into the ‘Connectors’ folder in the ‘My Tableau Repository’ folder in your documents. This allows the creation of a custom connector that enables Tableau to communicate with the MongoDB Atlas product.

Create a new database connection in the desktop app and search for ‘MongoDB Atlas by MongoDB’ under ‘Installed Connectors’. Authenticate with the connector using the credentials set for your Organization using the Viam CLI.

• MongoDB URI: ‘mongodb://<YOUR DATA FEDERATION HOSTNAME STRING>’
• Database: ‘sensorData’
• Username: ‘db-user-<YOUR ORG ID>
• Password: Whatever password you set when configuring the database.

You can then choose to import all of the data from your organization, or you can filter down to specific parameters such as a particular machine or sensor component if you want to narrow down the data included in Tableau.

‍

When you have all your desired data loaded into Tableau with a live data source, you can begin organizing your data by splitting sensor readings into different table rows, renaming your readings, and organizing them into tables however you want. Once you have organized your data sources into different sub-tables, it is as simple as dragging and dropping the desired sub-tables into different visualization templates.

‍

Once the data is imported into Tableau, the next step is to design visualizations that convey meaningful insights. You can consider different visualizations for different types of data interpretation:

‍ Time-Series Analysis: Line charts are the go-to for spotting trends and patterns over time. This is particularly useful for monitoring sensor data such as temperature, humidity, or motion over specific periods.

‍

‍

Interactive Dashboards: Build interactive dashboards that let users dig into specific data points, apply filters, and see real-time updates. Dashboards can make your data more engaging and help you dive deeper into the analysis at a glance.

‍

‍

Heatmaps and Scatter Plots: Want to spot correlations and anomalies? Heatmaps and scatter plots can be the answer. A heatmap can show you the intensity of sensor readings across different areas, while a scatter plot might uncover relationships between various sensor data types.

‍

The benefits of using Tableau for sensor data

By visualizing Viam data in Tableau, data scientists can turn a data dump into valuable insights that drive smarter decision-making and more. Tableau’s extensive customization options let you tailor visualizations to specific needs and preferences, ensuring the most relevant insights are front and center for analysis.

Additionally, thanks to Tableau’s live data connections, you can keep an eye on sensor data in real-time, allowing for quick responses to critical changes as your machine is collecting data in the real world. Get started with visualizing important machine data with Viam today.

Originally published at https://www.viam.com.

Using Tableau to visualize sensor data in the real world was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

If your dog is as insatiable as mine, you are familiar with having to wake up every morning to the sound of gentle whining at the door and the pitter patter of begging paws on the floor — two hours before the alarm. The sun has barely risen on the horizon as you glance out your east facing window, and you can see a moist nose peering under your door frame. Your dog commands you: it’s time to eat.

To allow myself to get some extra zzz’s before work, I’ve built a robot to feed my dog in the morning. It has also come in useful to give him some treats for being a Good Boy™ while I’m spending my day at the office.

In this tutorial you can follow along and build your own pet feeder. You will use the Viam app’s Data Manager to train a custom machine learning model that recognizes your pet and use the Machine Learning Service and the vision service to use it on your robot. The final component is a stepper motor and a 3D printed model which holds and dispenses treats when your pet is recognized.

Hardware

You will need the following hardware components:

1. A computer running macOS or Linux
2. Raspberry Pi with microSD card (and microSD card reader), with viam-server installed following the Installation Guide.
3. Raspberry Pi power supply
4. Stepper motor and motor driver
5. 12V power supply adaptor for motor driver
6. Simple USB powered webcam
7. Assorted jumper wires
8. Four 16mm or 20mm M3 screws

Tools and other materials

You will also need the following tools and materials:

1. Wide mouth Mason Jar or blender cup (if you want to avoid using glass!)
2. Small pet treats or dry kibble
3. Tools for assembly such as screwdrivers and allen keys
4. 3D printer (or somewhere you can order 3D printed parts from)
5. 3D printed STL models, wiring, and configuration recommendations.

Software

You will need the following software:

• Python 3
• pip
• viam-server installed to your board. If you haven’t done this, we’ll walk you through it in the next section.

Assemble your robot

The STL files for the smart feeder robot are available on GitHub.

There should have been a video here but your browser does not seem to support it.

1. Mount your Raspberry Pi to the side of the main body of your pet feeder using the provided mounting screw holes.
2. Connect your power source to the Pi through the side hole.
3. Mount your webcam to the front of your pet feeder. Connect the USB cable to your Pi.
4. Insert the 3D printed stepper motor wheel into your pet feeder. This is what will funnel treats out of your pet feeder programmatically.
5. Place your stepper motor into the motor holder part and gently slide the wires through the hole that leads through the body of your feeder and feeds the cables out on the Raspberry Pi side.
6. Slide the motor driver holder into the body of your feeder, it should sit flush and fit nicely.
7. Connect your stepper motor to the motor driver according to this wiring diagram:

Configure and test your robot

1. If you haven’t already, set up the Raspberry Pi by following our Raspberry Pi Setup Guide.
2. Go to the Viam app and create a new machine instance in your preferred organization.
3. Then follow the instructions on the Setup tab.

Now that you’ve set up your robot, you can start configuring and testing it.

Configure your board

Head to the Config tab on your machine’s page. Click on the Components subtab and click the Create component button in the lower-left corner.

Select board as the type and pi as the model. Name the component pi, then click Create.

Configure your webcam

Click Create component and add your webcam with type camera and model webcam. Name the component petcam, then click Create.

Click on the video path. If the robot is connected, a dropdown menu with available cameras will appear. Select your camera.

TIP

If you are unsure which camera to select, select one, save the configuration and go to the Control tab to confirm you can see the expected video stream.

Configure your stepper motor

Finally, click Create component and add another component with type motor and model gpiostepper.

1. If you used the same pins as in the wiring diagram, set the direction to pin 15 GPIO 22, and the step logic to pin 16 GPIO 23.
2. Set the Enable pins toggle to low, then set the resulting Enabled Low dropdown to pin 18 GPIO 24.
3. Set the ticks per rotation to 400 and select your board model, pi.

Click Save config in the bottom left corner of the screen.

To test everything is wired and configured correctly, head to the Control tab. Start by testing the motor. Click on the motor panel and set the RPM to 20 and # of Revolutions to 100 to see your treat dispensing mechanism in action. Feel free to tweak these values to achieve the desired speed of your dispenser.

Next, test your camera. Click on the camera panel and toggle the camera on. Now check if you can see your pet! Your pet may be a little skeptical of your robot at first, but once you get some treats in there, your furry friend will love it in no time!

Use machine learning to recognize your pet

Let’s make our pet feeder smart with some data capture and machine learning models! To do that, you’ll first have to configure Data Management to capture images. Then you can use these images to train a machine learning model on your pet.

Configure data management

To enable the data capture on your robot, do the following:

1. Under the Config tab, select Services, and navigate to Create service. Here, you will add a service so your robot can sync data to the Viam app in the cloud.
2. For type, select Data Management from the dropdown, and give your service a name. We usedpet-data for this tutorial.
3. Ensure that Data Capture is enabled and Cloud Sync is enabled. Enabling data capture here will allow you to view the saved images in the Viam app and allow you to easily tag them and train your own machine learning model. You can leave the default directory as is. This is where your captured data is stored on-robot. By default, it saves it to the~/.viam/capture directory on your machine.

Next, enable the Data Management service on the camera component on your robot:

1. Go to the Components tab and scroll down to the camera component you previously configured.
2. Click + Add method in the Data Capture Configuration section.
3. Set the Type to ReadImage and the Frequency to 0.333. This will capture an image from the camera roughly once every 3 seconds. Feel free to adjust the frequency if you want the camera to capture more or less image data. You want to capture data quickly so that you have as many pictures of your pet as possible so your classifier model can be very accurate. You should also select the Mime Type that you want to capture. For this tutorial, we are capturingimage/jpeg data.

Capture images of your pet

Now it’s time to start collecting images of your beloved pet. Set your feeder up near an area your pet likes to hang out like your couch or their bed or mount it temporarily over their food bowl, or even just hold it in front of them for a couple of minutes. You can check that data is being captured by heading over to the DATA page and filtering your image data to show just images from the location your pet feeder is in. Capture as many images as you want. If possible, capture your pet from different angles and with different backgrounds. Disable Data Capture after you’re done capturing images of your pet.

Create a dataset and tag images

Head over to the DATA page and select an image captured from your machine. After selecting the image, you can type a custom tag for some of the objects you see in the image and you add it to a dataset. The first thing you want to consider is what tags you are trying to create and how you want your custom model to function.

For the treat dispenser, you can tag images with the name of the pet, in our case toast. Notice that in our image collection, we captured images at different angles and with different background compositions. This is to ensure that our model can continue to recognize the object no matter how your robot is viewing it through its camera. To be able to train on the data you are tagging you also need to add each image to a dataset.

Begin by selecting the image you would like to tag, and you will see all of the data that is associated with that image. Type in your desired tag in the Tags section.

Be mindful of your naming as you can only use alphanumeric characters and underscores: this is because the model will be exported as a .tflite file with a corresponding .txt file for labeling.

There should have been a video here but your browser does not seem to support it. There should have been a video here but your browser does not seem to support it.

Then use the Datasets dropdown to create a new dataset and assign the image to it. We called our dataset petfeeder.

For each image, add tags to indicate whether it contains your pet and add the image to your dataset.

Note we are just tagging the whole image as we are training an image classification model.

Continue parsing through your collected data (in this case images) and tag away and assign to your dataset to your heart’s desire. Tag as many images with as many tags until you are happy with your dataset. This is important for the next step.

View your dataset

Upon completion of tagging your data set, you can view the data in your dataset by clicking on your dataset’s name on the image sidebar or on the DATASETS subtab.

Train a model

From the dataset view, click on Train model, name your model and select Single label as the model type. Then select the label or labels that you used to label your pet images. We called it petfeeder and selected the tag toast and no-toast to train on images of the pup and images that do not contain the pup.

If you want your model to be able to recognize multiple pets you can instead create a Multi Label model based on multiple tags. Go ahead and select all the tags you would like to include in your model and click Train Model.

Deploy your model to your robot

Once the model has finished training, deploy it by adding an ML model service to your robot:

1. Navigate to the machine page on the Viam app. Click to the Config tab, then select the Services subtab.
2. Click Create service in the lower-left corner.
3. Select ML Model as the type, and select TFLite CPU as the model.
4. Enter puppyclassifier as the name, then click Create.
5. To configure your service and deploy a model onto your robot, select Deploy Model On Robot for the Deployment field.
6. Select your trained model (petfeeder) as your desired Model.

Use the vision service to detect your pet

To detect your pet with your machine learning model, you need to add a vision service that uses the model and a transform camera that applies the vision service to an existing camera stream and specifies a confidence threshold:

1. From the Services subtab, click Create service in the lower-left corner.
2. Select Vision as the type and ML Model as the model.
3. Enter a name for your ML model service and click Create.
4. Select the model you previously created in the dropdown menu.
5. Navigate to the Components subtab and click Create component in the lower-left corner.
6. Create a transform camera by selecting type camera and model transform.
7. Enter classifier_cam as the name for your camera, then click Create.
8. Replace the JSON attributes with the following object which specifies the camera source for the transform cam and also defines a pipeline that adds the classifier you created.

{
"source": "petcam",
"pipeline": [
  {
      "attributes": {
          "classifier_name": "puppyclassifier",
          "confidence_threshold": 0.9
      },
      "type": "classifications"
  }
]
}

9. Head to your robots Control tab, click on your transform cam, toggle it on and you should be able to view your transform cam and if pointed at your pet, it should show it detecting your pet!

Control your robot programmatically

With your robot configured, you can now add a program to your robot that controls the pet feeder when executed, using the Viam SDK in the language of your choice. This tutorial uses Python.

Set up your Python environment

Open your terminal and ssh into your Pi. Run the following command to install the Python package manager onto your Pi:

sudo apt install python3-pip

Create a folder named petfeeder for your code and create a file called main.py inside.

The Viam Python SDK allows you to write programs in the Python programming language to operate robots using Viam. To install the Python SDK on your Raspberry Pi, run the following command in your existing ssh session to your Pi:

pip3 install --target=petfeeder viam-sdk python-vlc

Important

If you want your robot to automatically run your code upon startup, it is important to install the package into the petfeeder folder because of how the Viam platform runs the process.

Add the connection code

Go to your robot’s page on the Viam app and navigate to the Code sample tab. Select Python, then copy the generated sample code and paste it into the main.py file.

API key and API key ID

By default, the sample code does not include your machine API key and API key ID. We strongly recommend that you add your API key and API key ID as an environment variable and import this variable into your development environment as needed.

To show your machine’s API key and API key ID in the sample code, toggle Include secret on the Code sample tab. You can also see your API key and API key ID on your machine’s Security tab.

Caution

Do not share your API key or machine address publicly. Sharing this information could compromise your system security by allowing unauthorized access to your machine, or to the computer running your machine.

Save the file and run this command to execute the code:

python3 main.py

When executed, this sample code connects to your robot as a client and prints the available resources.

Add the logic

If your program ran successfully and you saw a list of resources printed from the program, you can continue to add the robot logic.

You’ll be using the puppyclassifier. The following code initializes a camera and the puppyclassifier and shows you how to get the classifications from the classifier when passing in the camera stream as an argument:

petcam = Camera.from_robot(robot, "petcam")
puppyclassifier = VisionClient.from_robot(robot, "puppyclassifier")
classifications = await puppyclassifier.get_classifications_from_camera(
    camera_name)

Remove the existing code in the main function and replace it with the following logic where the the code gets classifications from the puppyclassifier based on the camera stream, and if a pet is found, activates the stepper motor using the go_for() method to move a certain amount to dispense a treat.

async def main():
    robot = await connect()
    # robot components + services below, update these based on how you named
    # them in configuration
    pi = Board.from_robot(robot, "pi")
    petcam = Camera.from_robot(robot, "petcam")
    stepper = Motor.from_robot(robot, "stepper")
    puppyclassifier = VisionClient.from_robot(robot, "puppyclassifier")
    while True:
        # look if the camera is seeing the dog
        found = False
        classifications = await \
            puppyclassifier.get_classifications_from_camera(camera_name)
        for d in classifications:
            # check if the model is confident in the classification
            if d.confidence > 0.7:
                print(d)
                if d.class_name.lower() == "toastml":
                    print("This is Toast")
                    found = True
        if (found):
            # turn on the stepper motor
            print("giving snack")
            state = "on"
            await stepper.go_for(rpm=80, revolutions=2)
            # stops the treat from constantly being dispensed
            time.sleep(300)
        else:
            # turn off the stepper motor
            print("it's not the dog, no snacks")
            state = "off"
            await stepper.stop()
        await asyncio.sleep(5)
        # don't forget to close the robot when you're done!
        await robot.close()

if __name__ == '__main__':
    asyncio.run(main())

Save your file and run the code, put your pet in front of the robot to check it works:

python3 main.py

Run the program automatically

One more thing. Right now, you need to run the code manually every time you want your robot to work. However, you can configure Viam to automatically run your code as a process.

Navigate to the Config tab of your machine’s page in the Viam app. Click on the Processes subtab and navigate to the Create process menu.

Enter main as the process name and click Create process.

In the new process panel, enter python3 as the executable, main.py as the argument, and the working directory of your Raspberry Pi as /home/pi/petfeeder. Click on Add argument.

Click Save config in the bottom left corner of the screen.

Now your robot starts looking for your pet automatically once booted!

• Add speakers and record your voice so that the pet feeder can play a message to your pet each time it dispenses a treat.
• Train an ML model to recognize when your pet performs a trick, and withhold the treat until a specific trick is detected.
• Add a button that your pet must press to access the treat. If you add several treat types, you might include a different color button for each treat type, allowing your pet to choose.
• If you have multiple pets, you could configure different treats for each pet by training the ML model on each pet, and dispensing different treats depending on the pet recognized.

Full code

import asyncio
import os
import time
from viam.robot.client import RobotClient
from viam.rpc.dial import Credentials, DialOptions
from viam.components.board import Board
from viam.components.camera import Camera
from viam.components.motor import Motor
from viam.services.vision import VisionClient
# these must be set, you can get them from your machine's 'Code sample' tab
robot_api_key = os.getenv('ROBOT_API_KEY') or ''
robot_api_key_id = os.getenv('ROBOT_API_KEY_ID') or ''
robot_address = os.getenv('ROBOT_ADDRESS') or ''
# change this if you named your camera differently in your robot configuration
camera_name = os.getenv('ROBOT_CAMERA') or 'petcam'

async def connect():
    opts = RobotClient.Options.with_api_key(
      api_key=robot_api_key,
      api_key_id=robot_api_key_id
    )
    return await RobotClient.at_address(robot_address, opts)

async def main():
    robot = await connect()
    # robot components + services below, update these based on how you named
    # them in configuration
    pi = Board.from_robot(robot, "pi")
    petcam = Camera.from_robot(robot, "petcam")
    stepper = Motor.from_robot(robot, "stepper")
    puppyclassifier = VisionClient.from_robot(robot, "puppyclassifier")
    while True:
        # look if the camera is seeing the dog
        found = False
        classifications = await \
            puppyclassifier.get_classifications_from_camera(camera_name)
        for d in classifications:
            # check if the model is confident in the classification
            if d.confidence > 0.7:
                print(d)
                if d.class_name.lower() == "toastml":
                    print("This is Toast")
                    found = True
        if (found):
            # turn on the stepper motor
            print("giving snack")
            state = "on"
            await stepper.go_for(rpm=80, revolutions=2)
            # stops the treat from constantly being dispensed
            time.sleep(300)
        else:
            # turn off the stepper motor
            print("it's not the dog, no snacks")
            state = "off"
            await stepper.stop()
        await asyncio.sleep(5)
        # don't forget to close the robot when you're done!
        await robot.close()

if __name__ == '__main__':
    asyncio.run(main())

Next steps

Take your smart pet feeder to the next level! You could try one of the following:

• Add speakers and record your voice so that the pet feeder can play a message to your pet each time it dispenses a treat.
• Train an ML model to recognize when your pet performs a trick, and withhold the treat until a specific trick is detected.
• Add a button that your pet must press to access the treat. If you add several treat types, you might include a different color button for each treat type, allowing your pet to choose.
• If you have multiple pets, you could configure different treats for each pet by training the ML model on each pet, and dispensing different treats depending on the pet recognized.

I have always been a fan of automation and home improvements, but time constraints and a reluctance to invest in off-the-shelf solutions that may not meet my specific needs held me back from exploring this interest further. What if there was an app that lets you build a smart device yourself, faster than it takes for a store-bought one to be shipped to your door?

In this blog post, you’ll find a list of DIY smart home projects using Viam, a smart device and robotics software, perfect for beginner robot building. They’re budget-friendly and easy to start with, especially if you’re just getting into robotics or have a hectic schedule.

Create your very own smart pet feeder using machine learning (link)

Imagine getting a few extra moments of sleep before you start your workday? With this diy robot, you’ll automate your pet’s morning meal and give them a few extra treats while away using computer vision and machine learning.

Difficulty Level: Easy

Viam Services & Components: Shopping List: Steps:

• Assemble all of your hardware components.
• Configure and test your robot in the Viam app.
• Put the pet feeder in front of your beloved pet and check your camera.
• Start collecting pictures of your pet in the Data Manager
• Create a dataset and tag all the pictures of your pet.
• Train a model on your pet dataset.
• Deploy your custom machine-learning model onto your robot.
• Control the pet feeder with some code.
• Watch your pet eat treats whenever the pet feeder sees your pet!

Make a facial verification system for home security (link)

‍

Transform your home security with a DIY facial verification system. Ditch the limitations of off-the-shelf options and create a system that recognizes family and friends, allowing them access and even sends text alerts for doorbell rings.

Utilize the machine learning capabilities of the Viam platform to build a smart security system with just a board and a camera. It will intelligently disarm the alarm if it identifies an approved face after detecting someone. Simple, effective, and uniquely yours.

Difficulty Level: Easy

Viam Services & Components: Shopping List: Steps:

• Create a new machine in the Viam app and install viam-server on your new machine.
• Configure your camera component and test it in the Control tab.
• Set up your security camera in your desired location in your home.
• Capture images of your family members and create a dataset for people you want to identify with your robot.
• Train a model on your dataset.
• Configure a facial detector using that trained model.
• Configure a verification system and configure a transform camera.
• Watch your verification system in action!

Make a drink-carrying robot for your house (link)

Getting up from your couch for a beverage is a thing of the past. Having a personal robot assistant that transports beverages, snacks, or pretty much whatever you want in between the rooms of your house is easier than you think.

With a rover base, camera, ultrasonic sensors, and batteries, you can have your very own smart drink-carrying robot in your home!

Difficulty Level: Intermediate

Viam Services & Components: Steps

• Set up your robot’s board and base
• Configure the camera and ultrasonic sensor
• Configure an ML Model Service to detect people and objects
• Set up a detection camera
• Write some robot logic that detects obstacles and moves the base around to your desired person
• Enjoy your robot-delivered treats

The integration of robots and smart machines into our homes is more than futuristic fantasy; it’s a current reality changing our daily lives.

Imagine the convenience of a drink-carrying robot that quenches your thirst, the security of a custom-tailored home system, or the ease of a smart pet feeder caring for your pet in your absence. These projects are not just fun but also great starting points for learning how to build simple robots with Viam.

Dive into the types of smart machines you can create and explore building your own robot. What will your first project be? Join Viam’s online community, draw inspiration from others, and share your creations to be featured on our socials!

Originally published at https://www.viam.com.

I spend most of my day now managing the developer community for Viam Robotics on Discord, and hanging out with robotics enthusiasts and software engineers have given me a new perspective on the work I do as a Developer Advocate. Typically, my role involves crafting tutorials and demos to highlight our product’s features. After spending time with our community, I have been reflecting on how I can create more robot demos that can engage not just people in a room or someone reading a blog post, but how do I create something a whole online community can play with?

So, here’s a thought that’s been buzzing in my mind: How easy would it be to create a Robot Bot? Discord Bots are very common and pretty easy to make yourself thanks to awesome open source documentation in the Discord Developer Portal. Viam is an open source framework for building and programming robotics with flexible SDKs, so it is easy to build other interfaces on top of them.

So here I am going to show you how to control a physical robot running viam-server, all through simple DIY Discord chat commands in less than an hour.

Set up a real physical robot.

1. Build a robot. Yes, like an actual physical robot. I built a Viam rover with a wheeled base for simple movements.

1. Prepare a board for viam-server installation. You can check out how on the Viam Docs site. I chose a Raspberry Pi for my robot. Create SSH credentials and log into your robot in your terminal.
2. Set up a new robot on app.viam.com. If you don’t have an account, it is free to use and easy to sign up.

1. Configure a robot, and test it in the control tab. I used a Viam rover, so setting up a pre-configured Fragment took just a few seconds. Head to the Control tab and make sure all your parts are working and you can move your base and see the camera.

Prepare your development environment.

I used Python, and to manage Python packages efficiently I recommend creating a virtual environment.

1. Create your project directory and make sure you are in the right folders. $ mkdir discordBot
   $ cd discordBot
2. Create your virtual environment and activate it.
   $ sudo apt-get install python3-venv
   $ python3 -m venv virtualenvironment
   $ source virtualenvironment/bin/activate
3. Install the necessary packages. I used viam-sdk and discord.py
   $ pip install viam-sdk
   $ pip3 install discord.py

Prepare your Discord Developer Configurations

1. Go to discord.com/developers/applications and select New Application. Name it and head to the Bot tab. I named this bot RobotBot (I know, very creative).

1. Enable Privileged Gateway Intents permissions for MESSAGE CONTENT INTENT. This means you’re allowing the bot to send and receive messages. Set desired bot permissions as well, I picked Send Messages.

1. Head to OAuth2 tab and select OAuth2 URL Generator
2. Select Bot, and add specific bot permissions (Send Messages, Send TTS Messages, Manage Messages, Attach Files)
3. Copy the URL and go to the URL in the browser.
4. Add to authorized server

Testing the Discord Bot

1. Go back to the Bot tab in the Discord Developer Portal and reveal your token and copy it.
2. Add to to the following script. This script is pretty boilerplate as a simple example of a Discord Chat Bot. I used environment variables to set the token.

import discord
import os 
from discord.ext import commands

discord_token = os.getenv('DISCORD_TOKEN')

intents = discord.Intents.default()
intents.message_content = True
bot = commands.Bot(command_prefix='/', intents=intents)

@bot.command()
async def hello(ctx):
await ctx.send('hello from robotbot')
bot.run(discord_token)

Run your code, your bot should go online in your Discord server.

Now say hello in the chat, your Discord bot will greet you back if the connections are all correct.

Add your robot controls to the Discord Bot using Viam’s Python SDK

Integrate Viam robot code into your discord bot. Create a bot command and corresponding robot movement. I grabbed the boilerplate code from the Code Sample tab in the Viam App, and removed components I was not using for the sake of this example. I am only using the Base API.

Here are some examples of methods in the Base API that can help simply control my rover. I am going to use the MoveStraight() method.

I’ve used Environment Variables to set my API Key and Robot Address which are unique to your robot. Those can also be conveniently found in the Code Sample tab.

Create a forward() function (or really any movement you want) following the same logic. This logic can apply to any robot functionality you want to write using a Discord Bot to control a Viam robot. Here I am describing what is happening in async def forward(ctx):

1. Use the @bot.command decorator to start off your function. These decorators are used to register functions as bot commands, and they can’t be nested inside other functions. They should be used at the top level in your script.
2. Create a variable in scope of that function for your component you want to control.
3. Write the command response for the bot to auto reply in discord.
4. Write the function that moves the robot.

import discord
import asyncio
import os

from viam.robot.client import RobotClient
from viam.rpc.dial import Credentials, DialOptions
from viam.components.base import Base
from discord.ext import commands


api_key = os.getenv('API_KEY')
api_key_id = os.getenv('API_KEY_ID')
robot_address = os.getenv('ROBOT_ADDRESS')
discord_token = os.getenv('DISCORD_TOKEN')

intents = discord.Intents.default()
intents.message_content = True
bot = commands.Bot(command_prefix='/', intents=intents)

async def connect():
  opts = RobotClient.Options.with_api_key(
    api_key,
    api_key_id
  )
  return await RobotClient.at_address(robot_address, opts)

@bot.command()
async def forward(ctx):
  robot = await connect()
  viam_base = Base.from_robot(robot, "viam_base")
  await ctx.send('i`m going forward')
  await viam_base.move_straight(distance=10, velocity=50)
  await viam_base.stop()

@bot.command()
async def hello(ctx):
  await ctx.send('hello from robotbot')
 
bot.run(discord_token)

Easy peasy. Run your code, and the bot will instantly go online. Type in your command in the chat

Your robot should be moving!

You can apply this logic to control any robot functions using Viam and a Discord Bot. This simple project was completed in under an hour, demonstrating the power of open source and the true flexibility of Viam’s SDK’s. For my next iteration of this project, I think it would be pretty fun and chaotic to let a server of thousands of people drive this rover around my house. Maybe I should create a SLAM map of my house and use Viam’s Motion Planning to make this project a bit more robust.

Let me know if you try this! You can find me and our amazing developer community on Discord — tag your projects in #built-on-viam to be featured on our socials.

To get a glimpse of how this was made, I interviewed the creators of this project-keep reading for more!

Tell us about yourself and your team.

We are Team Spyder! We’re made up of four 3rd year engineering students at the University of Waterloo who are experienced in both engineering competitions and hackathons. This year, we set out to compete in Hack the North 2023 for fun, learning something new, and this year, a win!

Can you walk us through the initial concept of your build?

The initial idea struck when leftover electronic parts and a toy car faced the trash at the student center. Why not turn this into something?

Inspired by Michael Reeves’ build, we fantasized about building a mind-controlled car, but aimed to make it even better.

Michael Reeve’s mind-controlled car: the inspiration behind Spyder.

We took the car apart to see if our idea was even feasible-can we modify this to match what we need? Would we have the right electronics to make this happen?
Turns out when you have a bunch of overexcited engineering students working on a project like this one, it doesn’t matter. We’ll make our own electronics.

So, what does your build do?

We made several mechanical and electronic modifications to the car to meet our specifications, including:

• Designing and 3D printing a custom rack and pinion gear to adjust the steering system, coupled with an additional motor for automated steering.
• Upgrading the battery system for enhanced performance and durability.
• Incorporating advanced electronics such as a Raspberry Pi 4, buck converter, LED controller, custom power distribution board made by Ari, and motor driver.

Next was the software component. We used Viam to control the car electronics once they were all plugged in and soldered in.
For the mind-control aspect, we harnessed the Neurosity Crown and its AI, training it with a dataset derived from Patrick’s brain waves. This approach enabled the car to respond to Patrick’s thoughts, moving forward when he intended it to.

Patrick using the Neurosity Crown to capture his brain signals.

We integrated the two software elements using scripts crafted by Melda, utilizing web sockets for connectivity.

Can you walk us through step-by-step on how you used Viam for your build?

We initially turned to Viam to validate our concept. Aaditya explored Viam’s documentation, employing a motor driver compatible with the Raspberry Pi. Following the guidelines provided, we connected it and leveraged the Viam server for system troubleshooting!

This step was crucial as it allowed us to confirm the integrity of our electrical system before writing any code. The Viam server was instrumental for our troubleshooting and sanity checks, saving us several hours.

It also enabled us to test our motors and mechanical systems to their limits. After validating our systems, we found the code generated by Viam and its SDKs straightforward to integrate into our codebase.

What were the most challenging aspects of your build?

One of the most challenging aspects was getting the right data from the Neurosity Crown. Since the electrodes sit on top of the scalp, it makes it difficult to capture clear brain waves.

Think about it like you’re listening to a conversation happening in a room but you’re outside the closed door. It’s muffled and sometimes you can’t quite get all the words.

Eventually, we got it to work after gathering a few datasets and Patrick’s thoughts were properly recognized!

What was the learning curve like for Viam’s platform?

Viam’s platform and SDKs were very easy to pick up and use in our design process and codebase. Viam’s interface was simple and easy to use, but still packed with useful features. The SDKs were very straightforward and a quick read through the documentation was all it took to figure out how to effectively use them.

Any plans to upgrade your current hackathon build?

We’re considering integrating a webcam for Computer Vision, enabling the car to automatically slow down upon detecting humans in its path.

We also aim to enable simultaneous recognition of multiple commands, enhance our electromechanical systems to support a passenger in the car, and even introduce mind-controlled unlocking of the vehicle!
We have a few other ideas to use a Viam Rover for next year’s Hack the North project so stay tuned for that!

Any final thoughts or words of wisdom you would like to share with aspiring builders and innovators?

Don’t be afraid to try something new! We initially started making this project as a joke. Who would ever think about creating a mind-controlled children’s car?

We had a lot of fun making this project and learning how to work with neuro-technology and Viam. The win is definitely the cherry on top of the whole experience, but ultimately, we came out of it as better engineers and created fun memories to go with it.

Things won’t always go to plan. We had quite a few hiccups and had to pivot a few times, especially at the beginning, but it’s important that you try your best and go with it. Sometimes when things don’t work out initially, they end up becoming better.

Ready to create your own smart machine? Dive into some Viam tutorials for immediate building, explore modular resources to enhance any project with Viam, and join the online community for endless inspiration from innovative creations!”

Originally published at https://www.viam.com.

The challenge for attendees was simple: Make something amazing. And did they deliver! We’ve put together the best bits for you to check out and get inspired in the process.

Entering Hack the North

As a sponsor of Hack the North, we hosted a category within the competition-Best Use of Viam-giving students the opportunity to win special prizes and see the versatility of our software. They could use the platform for hardware configuration, software engineering, or a combination of both for our category.

To introduce them to the software, our team gave an API workshop titled “How to Bring Your Robotic Projects to Life,” which taught hackathon participants how to leverage the Viam platform to make their machines smarter.

Through this demo, we:

• Introduced “Tipsy,” our mobile rover, trained in computer vision and machine learning for autonomous navigation.
• Created a fun, Pac-Man-inspired demo with team members in unique shirts, triggering Tipsy’s chase or escape.
• Demonstrated Viam’s machine learning essentials, teaching students to train their models within the app.
• Showcased how to tailor Computer Vision detections through coding and configuration.
• Conducted a live coding session exploring our SDKs and real-time robot configuration.
• Highlighted the Viam app’s seamless functionality, impressing students with its user-friendly interface.

‍
The aftermath? Students and teams were inspired to use our platform to accelerate their projects-building smart machines faster within the hackathons short timeline.

With over 30 inquiries for Viam Rover development kits, six lucky groups were allowed to prototype using our rovers to help them realize their hackathon goals. See some of the highlights below.

Seeing the projects come to life

Use the Force… or just one of our SDK’s

Imagine using your thoughts to control a car-could that just be a Jedi mind trick? Or is it simply just the magic of open source integrations?

Spyder, aptly named after the toy car Audi the group reimagined, uses the Viam Golang SDK and uses the Neurosity Crown to take the brainwaves of an individual, train an AI model to detect and identify certain brainwave patterns, and output them as a recognizable output to humans.
The team collected brain electrical impulses, and forwarded those commands to the Viam interface to control the steering of the car. It makes you wonder, “could this be the future of autonomous vehicles?”

Dive deeper into this project through their Devpost and source code.

Another set of eyes for teachers in classrooms

Meet Rezbot: your classroom’s intelligent ally streamlining daily tasks for teachers. From taking attendance to monitoring engagement, it keeps a finger on the pulse of every student’s presence and participation, transforming the way educators prepare lessons and assessments.

Rezbot is a web application that uses Computer Vision to identify students and automate the process of attendance. It can simulate dashboards for teachers to view statistics on certain students’ classroom habits.

This group used our Flutter SDK to create a web app where teachers can access a live camera stream from their Viam robot and collect image data using Computer Vision to feed data into these dashboards.

For more information on Rezbot, head to the team’s Devpost.

Robotic paparazzi for our star hackers

Next up on our roster of awesome projects that use Viam, Pic Perfect is the friendly robot companion here to replace that friend who always takes the worst Instagram photos.

Using a facial detection model, this smart machine tracks your every move, adjusting the robot to the ideal angle and distance so the subject is perfectly centered in the frame. With a simple thumbs-up gesture, it takes your photos and applies filters to produce perfectly crafted portraits which are then shown on a web app.

To build this smart machine, the group leveraged Viam’s Python SDK and used a Viam Rover as the base.

Discover more about this build through the team’s Devpost.

Awarding the winners of our category

As the weekend drew to a close, students took to the stage, showcasing their dedication and innovation during this intense hackathon.

Viam proudly presented the award for the “Best Use of Viam.” The winners, each receiving a fully equipped Viam Rover complete with a Raspberry Pi, a LIDAR, and an assortment of hardware, demonstrated exceptional skill and creativity.

We honored two victors in our category: “Spyder,” for its ingenious application of open-source technologies in tandem with Viam, and “Pic-Perfect,” for skillfully integrating machine learning models into their rover prototype and developing a fully functional front end for their robotics project.

Reflecting on the event

Thinking about the weekend, one thing became crystal clear: Viam’s platform, with its vast language support and flexibility, is an enabler of creativity.
Many students were surprised at how many languages we support, and the fact that projects were submitted utilizing different SDK’s is proof of just that. It showed that it doesn’t matter what type of engineer you are: there’s something for everyone on the Viam platform.

So, what’s the takeaway for you? Like the students at the hackathon, embrace the challenge of a timeline and try to build a fully functional project in just a weekend- utilizing Viam to help of course. Dive into our tutorials and documentation to get started today.

Originally published at https://www.viam.com.