Scattered sticky notes and paper? Digitize and organize your handwriting with Nuwa Pen.
Latest Updates from Our Project:
Going Back & Moving Forward: Part 4/4
over 1 year ago
– Fri, Jun 02, 2023 at 11:48:44 AM
Hey backers,
We hope you had a great week. The campaign is ending soon (4 days!!), so we’re near the end of the Going Back & Moving Forward series. But don’t worry; we’ll update you regularly about the development as we go. This time we’ll show you the progress we’ve made on the cameras in PenUI (our internal development software) and explain why we’ve opted for proprietary machine-learning algorithms over OCR. For now, here’s the overview for this week! If you haven’t already, check out Part 1, Part 2, and Part 3.
Collaborations: we like <3
Going Back: PenUI Part 2 - Cameras
Moving Forward: Introducing Nuwa Pen Case, Integrations & ML vs OCR
Collaboration: we like <3
Upgraded LeMat Magnetic Fixer 2.0
We would like to share a project that caught our attention, and we are excited to announce a collaboration between Nuwa Pen and LeMat Magnetic Fiber. We think this is a good match! LetMat will help you organize your gadgets, including your Nuwa Pen.
We encourage you to back them and keep your desk modernized and Nuwa Pen reachable! Click on the image for more info:
OENKLEN is a revolutionary home-use device to upgrade your oral care routine! Using the ozone water from mouth washing to combining with flossing, cleaning your braces... It will revolutionize your lifestyle in protecting your teeth.
5S Sterilization 99%
Breath Freshening
Alcohol-free
Child & Elderly Friendly
100% Natural & Organic
Patented Electrolysis Tech
Smart App
Going Back: PenUI Part 2 - Cameras and ML vs. OCR.
In our last segment, we shared more about how PenUI shows what Nuwa Pen writes, making it easier for us to continue optimizing the algorithms. The second component of this development is to make sense of what is happening in the cameras. There were large steps we needed to take to get to a small form factor for our tiny microcontroller to process these feeds and fuse all sensor data in real time. In previous updates, we showed what we could see with one camera. Now, we can collect more information by having 3 cameras rather than only 1.
The 3 micro cameras allow us to have 360-view and capture the handwriting from multiple perspectives. Then we use a Computer Vision algorithm to process these camera images. We combine Computer Vision and Sensor Fusion algorithms to digitize the strokes with maximum accuracy. Then you’ll see an exact copy of what you wrote on paper. After the cameras capture your writing, you can convert your handwriting to typed text or search through their content in the app. Which brings us to OCR.
For that conversion to take place, most would suggest Optical Character Recognition (OCR), but OCR works better with typed text; however, our focus is to recognize handwriting, and OCR is not the best candidate for that. Billions of people have different types of handwriting, which we will only be able to cover some of by using OCR. Instead, we take a step back to understand the data from our sensors: cameras, motion, and pressure sensors. The cameras provide a very close view of the letters while you write, and the motion sensor understands the path. We can use this information to detect what letters are being written, enabling us to create a state-of-the-art algorithm that will improve over time, and we can’t wait to get there.
You’re now up to speed about the journey of the pen and all the decisions we’ve made to build the pen of the future. We’re constantly working to improve the computer vision algorithms, optimizing the sensor fusion system, and updating our handwriting machine learning algorithms - and we’ll continue to share our updates with you. Please let us know in the comments what you think!
Moving Forward: Introducing Nuwa Pen Case, Integrations, and our international backers.
First up is the updated version of the Nuwa Pen Case. We’ve made some changes to the design, incorporating the tools in small drawers that you can tuck away and a small led status indicator that shows how many charges are still in the case. The picture below shows a 3D-printed prototype of the case with the drawer pulled out for the D1 refills.
This other picture shows the case renders from a few angles.
One we’ve all been waiting for a long time—a list of possible integrations, which we will be rolling out starting the first half of 2024. We’ll take your feedback on the following integrations while we roll out and develop a roadmap together. With your support, Nuwa Pen will be the most useful pen in the world through a series of powerful applications. Without further ado, here’s the list!
We also would like to share with you some entities that back us and believe in our project, such as the European Regional Development Fund, Samenwerking Noord Nederland, Consumer Electronics Show, and Philips Innovation Award.
It’s been an epic journey to go back in time and look through the Nuwa Pen project. It’s also been a pleasure to write this series for you. We’re nearing the end of our Kickstarter campaign, but we will continue accepting pre-orders through www.nuwapen.com. If you know someone who needs a Nuwa Pen, feel free to share the website!
Hope you’ve liked this series. Talk to you again soon :)
Warm regards,
Marc, on behalf of the entire Nuwa Pen Team
Going Back & Moving Forward: Part 3/4
over 1 year ago
– Tue, May 30, 2023 at 09:10:18 AM
Hello again backers,
We hope you had a great weekend. As promised, we are continuing the Going Back & Moving Forward series. This time we’ll show our team and give a sneak peek into our internal software. Here’s the overview for this week!
Going Back: The Team and PenUI
Going Forward: Demo Searchability and Blowout Explanations
Going Back: The Team and PenUI
In our last update, we shared how we got to such a small form factor (see the picture below). This took quite a long time. There’re a few iterations in between and lots of small proof of concepts to connect the different parts of the pen, but maybe we’ll save those for a later documentary. For now, I wanted also to introduce our entire team. And here we are.
From left to right, you’ll see, Jiwoo (Computer Vision Engineer), Breno (Embedded Software Engineer), Anna (Marketing Designer), Frank (Computer Vision Engineer), Fiorella (Brand Marketer), Hilde (Community Manager), Marc (Chief Pen Salesmen), Shubham (Operations Lead), Mohamed (Machine Learning Engineer), Vishal (Lead Systems Engineer), Boyd (Hardware Engineer), Stefan (Finance), Mihai (Industrial Design Engineer) and Denis (Embedded Software Engineer). Missing from this picture are Julia (Lead Industrial Design Engineer), Cas (Design Engineer), and Mark (Industrial Design Engineer). Together, all 17 of us are working our butts off to bring the pen of the future to the palm of your hand. We specialize in what we do, bringing a century of cumulative knowledge and experience.
So you’ve seen what we’re making and are part of our mission. If not, I invite you to join us on our mission.
But how does it work?
If you’re also reading this Apple-r, Logitech-r, Meta-r, Microsoft-r, or Google-r - reach out to me at [email protected] if you see an opportunity to learn from each other - because this is absolutely insane what we’re doing here, and you know it.
Let’s start with a high-level overview of the algorithms, starting with the sensors. We’re incorporating inertial measurement units, cameras, and pressure sensors. With this data, we’re able to understand the orientation relative to the globe, see what you’re writing, and understand when you’re writing. This combination is fused in one incredible advanced sensor-fusion algorithm and then processed on a small 5x5mm energy-efficient microcontroller - the result is incredible.
To understand the output from the Nuwa Pen, we built our tool, which we call PenUI. This software runs on all our computers and helps us program and develop the software in real-time. The tech stack in the developer version pen and the actual Nuwa Pen are the same. As seen in the GIF below, the developer version of the pen with cables is very handy for internal testing as it is easier to debug the electronics and embedded software. Below is what processing the data on our developer (internal) Nuwa Pen looks like. What you’re looking at are three graphs that give us the Y, X, and Z orientation of the pen and see the pen move in real-time - in 3D.
The scribbles are digitally generated as we write on paper and help us continue to fine-tune the handwriting algorithms.
Moving Forward: Demo Searchability and Blowout Explanations
First, another demo! In the last posts, we showed you how we added dates to calendars and converted handwriting to typed text. Now, we want to show you how easy it’s to find back notes you love. As always, a disclaimer: All of our software is still under development so you can expect huge improvements over time.
Next are two blowouts of the Nuwa Pen internals. The first is the TRIDENT, where you’ll see the meniscus lens, which keeps dust from entering the device, and the tri-camera module to capture your strokes from all angles. The second image is a blowout of the entire pen, where you’ll see how the TRIDENT goes into the shaft alongside the printed circuit board, pressure sensor unit, battery, and end cap. Oh, right, also the standard D1 refill!
So much tech in a small place. Nuwa Pen is 11mm in diameter. Here’s a quarter-dollar coin for comparison.
This was Part One of getting to know PenUI - primarily focusing on the movement of the pen and a showcase for you to see how we’ve moved forward. In the next update, June 2, we’ll show what the camera sees through PenUI and go more in-depth about how our handwriting recognition works and why we chose to develop a proprietary machine learning algorithm over traditional OCR technology. You’ll also get a list of integrations we intend to introduce in early 2024!
Again, this is all incredibly exciting, and we really appreciate you all for taking the time to read, comment and follow this adventure. With your help, we really are making the most useful pen in the world. Please comment to let us know what you think, and see you again on June 2!
Warm greetings from the Netherlands,
Marc and The Nuwa Pen Team
Stretch Goal UNLOCKED + In Depth Series Part 2/4
over 1 year ago
– Thu, May 25, 2023 at 11:41:38 PM
Hello backers!
Much to celebrate, not only Part 2 of our Going Back & Moving Forward Series but also that we’ve passed over $150,000 in funding, which means that you’ve all unlocked lifetime access to Nuwa Pen+❗
We’re always curious about what you think, so feel free to leave your thoughts in the comments below. Here’s the overview for today:
🔓Free Lifetime Subscription Unlocked🔓
Going Back: The First Portable Nuwa Pen
Moving Forward: Demo of Text Conversion & Adding a Date
🔓Free Lifetime Subscription Unlocked🔓
Advanced Nuwa Pen+ features like Augmented Notes, Text Conversion, and App Integrations now come without any additional cost to you forever. Augmented Notes allows you to turn your handwriting into actionable snippets that you can port into other applications, like in the demo we’re showing further down in this update.
These features are what make Nuwa Pen the most useful pen in the world. It can just be you, paper and Nuwa Pen - and once you’re ready - you can sync your work and thoughts straight into your favorite apps. Thank you so much for bringing us so far; what should the next stretch goal be?
Going Back: The First Portable Nuwa Pen
As we saw in the last piece, which lasted until December 2021, you learned how we went from grabbing the first images - to realize we needed to correct for orientation - to having the first handwriting binarized into a computer. This piece will bring us to April 2022, when we started to develop all this technology into a form-factor that we could take with us wherever we wanted to go: Meet Cucumber 2.
Cucumber 2 Prototype (Q1 2022)
Cucumber 2 Camera Set Up
Cucumber 2 was a prototype that we could bring around to showcase our unique solution for writing on any paper. Cucumber 2 worked on a development board with two cameras on the tip. This gave us sight above and below the tip, as we needed to come up with a solution to see what was otherwise blocked by the pen tip. Hint: In the final product, we realized we needed three to get a 360-degree view. We even had a concept where the cameras would be at the top, but the user’s hand would block most of the useful information, and we couldn’t get a high-resolution but small-size camera that could detect handwriting from so far away.
Old Concept Of Bird's Eye View
We then had all of this technology but needed to fit it into a much smaller form factor. But we were also in the midst of a chip crisis, which meant we had to think from what was available to what would make it work. This much smaller form factor we called Asparagus. (We continued with green fruits & vegetables)
This was an incredibly difficult challenge, also because we had so many components to fit on this tiny PCB. Simple things like capacitors had lead times of 100+ weeks, but we managed. We worked very tightly together with our embedded software team and suppliers to see what was possible with the available components. We partnered with leading firms in low-power microprocessors and image sensors to develop a state-of-the-art composition of components tailored to making Nuwa Pen. This reduced the size from a large box to a pocket-size device. You can see in the picture below our first iteration of Asparagus, with another banana for scale.
Cucumber 2, A Banana and Asparagus 1
This series will continue to offer you a peek into the creation of the world's first AI-powered Smart Pen. In the next episode, shared on May 29, dive deeper into our unique software, PenUI for internal use, and the power of searchability. In future posts, discover the compelling reasons behind our choice of proprietary machine learning, and also learn about the unveiling of the Nuwa Pen's future integrations. We can't wait to share more!
Moving Forward: Demo of Text Conversion & Adding a Date Number
Here’s another live demo of Nuwa Pen converting a quick written note, and how we added the date to our calendar: from writing on paper to synchronizing on the app! Even though 5 was picked up a bit skewed, our handwriting recognition engine still converted it successfully - more on our proprietary engine on part 4/4 in this series. Heads up: Nuwa Pen (+Software) is still in development, so expect lots of improvements in the upcoming months - we’re very excited that you decided to join our journey, and if not, fasten your belts and join us!
See you at our next update, launching May 29. Very curious about what you think so far. Please let us know in the comments below!
New In-Depth Series: Going Back & Moving Forward
over 1 year ago
– Mon, May 22, 2023 at 10:00:48 AM
Dear backers,
Starting this week, we’re introducing a series called Going Back & Moving Forward where we’ll show more on the work that has been done to get to our campaign and the development we’re accomplishing as our campaign continues to soar!
We’d love to receive your comments on what you’d like to see next, but without further ado - here’s the overview for this update:
Last call for Super Early Bird
Going Back: From Ergonomics to The Second Concept
Moving Forward: Date Augmented Notes & Patent Filing
Early Bird - Ending Soon!
Today is the last day to get the super early bird discount. Starting tomorrow, backers will pay $30+ extra for their Nuwa Pen reward. Get it while you can!
Going Back (to 2021): From Ergonomics to The Second Concept
We started Nuwa Pen R&D in March 2021, and this part of the series lasted until December 2021.
For an instrument that’s been used by billions in the last thousand years, it was important to us that our pen of the future had to both be comfortable and feel precise. The shape, position of the grip, diameter, length, weight, balance, texture and material are all important to consider when developing a pen. Below are months of our own research consolidated in one graph.
Nuwa Pen customers are primarily note-taking users and not technical drawers so we focused on comfort over precision and came out in just over 11mm in the shaft diameter.
(c) Nuwa, 2021
This resulted in quite a few models, with the ones below being some of our favorites to hold. We didn’t focus too much on the components yet - we just wanted to make sure we had built a pen ergonomically sound that users could write for hours at a time.
Focus on Ergonomics
But now that we had the ergonomics and user experience in hand, it was time to consider how we work backwards to the technology. So one of the first things we did was look for small commercial cameras. By this time we estimated that we needed 2mm, but the smallest available at that time was 8mm. These cameras were housed on a camera-enabled earwax remover! So we bought a couple, and strapped them to a pen. It provided the first insights into understanding how far a camera would need to be away from a pen tip in order to get a good image of the letters.
Pen with Camera Strapped on Top
Testing Camera On Handwriting
What we noticed after making lots of recordings, is that the way the user held the pen altered the images we were getting in. If you held the pen a bit skewed, then the letters would look longer for example. To have our software get a better understanding of this, we knew we needed an inertial measurement unit to understand the movement and orientation of the pen. In came the second proof of concept, which housed a camera, IMU and pressure sensor to activate the groundbreaking recording device. This unique combination of sensors gave birth to the origin of our patent pending TRIDENT system. Isn’t it beautiful? This was in the summer of 2021.
Project Cucumber 1
We had to develop our first orientation algorithms, from scratch, to understand how the pen was moving in real time. This way our computer vision algorithms could always get a flat surface of the paper to analyze. The very first conceptual results are shown below, we could binarize images from a camera and correct for orientation of the images. This was the first steps of TRIDENT - our innovative camera system. We thought it was brilliant!
Embedded Orientation Alignment
First Image Binarization
The next step was to get it in a portable device, but we were in the middle of a chip crisis, so more on how we took on that challenge in the next part of Going Back & Moving Forward.
This series will continue to offer you a peek into the creation of the world's first AI-powered Smart Pen. As the series progresses, delve deeper into the minds behind the invention, our unique internal software, PenUI, and the power of searchability. Discover the compelling reasons behind our choice of proprietary machine learning to the unveiling of Nuwa Pen's future integrations. We can't wait to share more!
Moving Forward: Date Augmented Notes & Patent Filing
Here’s a live demo recorded in the office of how we added a handwritten date straight to our own calendar. Next up is going from your phone number to your contacts. All software is in development, and we'll update you about the improvements along the way.
We’ve also successfully filed our patent for Nuwa Pen! This means we can officially smack this label on our product now too.
This has been, is and will be an incredibly exciting journey. Very curious what you think so far, please let us know in the comments below!
Warm regards,
Marc and the Nuwa Pen Team
Lifetime membership for all backers ❤️ ¦ Nuwa Pen
over 1 year ago
– Wed, May 17, 2023 at 08:28:49 AM
Hey there!
We have some incredible news to share with you!
Thanks to your unwavering support, our Kickstarter campaign has been a resounding success. From reading all your comments, we realize that we should show our appreciation to you, so we have decided to change our original plan.
When we reach the $150K milestone on Kickstarter, we will offer a🎁 FREE lifetime membership to every Nuwa Pen subscriber instead of the one-year membership currently included in each pack!
And access the latest and greatest features without worrying about paying for upgrades or renewing your subscription.
But wait, there's more!
As a way of saying thank you, we're excited to introduce our Referral Program on Kickbooster.
Help us spread the word about Nuwa Pen by referring your friends, and you'll earn 10% cash back for each friend who backs our campaign! So, let's work together to reach our first stretch goal of $150K and make Nuwa Pen an even bigger success!
