Event camera #83

gerni17 · 2021-02-28T23:42:50Z

This is the event camera implementation.

* workin pipeline, wrong velocity format and of doesn't seem to work * nice pipeline with histo, but rather slow * check the framerate * not working pipeline * working pipeline of firt event sensor * commit before implementing trajectory * able to send the events to client * able to transfrm it ina image but not properly working * wrong deserialization * working pipeline * added timestep * not working trajectory * not working * working but not moving * some working * racing is workin properly, the oter one not really * working in racing * first working pipeline, but not ood yet * not working pieline * working but wrong error * not workin testing * having big errors again * events and other thing do not have same coordinates * working with second last images * nothing important * check coroutines * added client evet settings * i don't undertand why but this works * added histogram and removed second image * rosbag publisher * not working * working pipeline * tested and storing rosbags * cleaned code and added recording script * new env, writing to rosbag, no pos wirting * saving image and events * save image * writing images and erasing zeros * proper time function * saving properly * huge error accumulation * little faster and testing is more independent, overall working * recieving all images * snaga * changed testin bag destination * changed to previous port numbers * added depth and int64, still ring trajectory * added snaga config * this looks nice for now * working but has to be cleaned and made smarter, still having the helper function * wrkin ad clean * for snaga to produce video * trajectory * higher res * slightl change of trajectory * recording definitive * major changes to make it adaptive * formatted * added linear depth * added adaptive time of elia * add rotation * added corect pos output * added speed variing * added trajectory * doubled vel * added new trajectory * added angle to pure translation * add fourth traj * trajectory five * sixth trajectory * seventh trajectory * cleaned code

yun-long · 2021-03-15T16:14:17Z

.gitignore

@@ -3,6 +3,9 @@ dist
 *.egg-info
 *.so
 __pycache__
+build
+
+flightros/params/local.yaml


is this line necessary?

yun-long · 2021-03-15T16:15:08Z

flightlib/include/flightlib/bridges/unity_bridge.hpp

@@ -96,7 +103,7 @@ class UnityBridge {
  int64_t u_packet_latency_;

  // axuiliary variables
-  const Scalar unity_connection_time_out_{10.0};
+  const Scalar unity_connection_time_out_{1000.0};


why using such a large time out

yun-long · 2021-03-15T16:17:58Z

flightlib/include/flightlib/bridges/unity_message_types.hpp

+  int output_index{0};
+  // eventcamera settings
+  float Cm{0.1};
+  float Cp{0.1};


what are Cm and Cp?
and why not using "Scalar" instead "float"

cp and cm are the thresholds of the event sensor.

yun-long · 2021-03-15T16:18:57Z

flightlib/include/flightlib/common/types.hpp

+// #include <cstdio>
+// #include <stdint.h>
+// #include <cinttypes>
+// #include <ze/common/transformation.hpp>


maybe you can delete these comments if they are no longer needed

yun-long · 2021-03-15T16:19:26Z

flightlib/include/flightlib/common/types.hpp

+// using uint16_t = ze::uint16_t;
+// using uint16_t =std::unit16_t;
+// using Time = std::int64_t;
+


also delete these comments if no longer needed

yun-long · 2021-03-15T16:19:38Z

flightlib/include/flightlib/common/types.hpp

+//   uint16_t y;
+//   Time t;
+//   bool pol;
+// };


same as above

yun-long · 2021-03-15T16:21:11Z

flightlib/include/flightlib/sensors/event_camera.hpp

+  bool setsigmaCm(const float sigma_cm);
+  bool setRefractory(const uint64_t refractory_period);
+  bool setLogEps(const float log_eps);
+  bool setImgStore(const bool img_store);


can we use "Scalar" instead of "float"?

yun-long · 2021-03-15T16:21:32Z

flightlib/include/flightlib/sensors/event_camera.hpp

+  bool setImgStore(const bool img_store);
+
+  bool changeTime(TimeMessage_t time_msg);
+  double getSecSimTime();


also why using 'double" here

because we need a higher resolution for this variable

yun-long · 2021-03-15T16:22:08Z

flightlib/include/flightlib/sensors/event_camera.hpp

+  cv::Mat createEventimages();
+
+ private:
+  Logger logger_{"RBGCamera"};


This is an event camera, right?

yes, my bad

yun-long · 2021-03-15T16:32:22Z

flightlib/src/bridges/unity_bridge.cpp

@@ -81,6 +82,7 @@ bool UnityBridge::sendInitialSettings(void) {
  msg << "Pose";
  // create JSON object for initial settings
  json json_mesg = settings_;
+  std::cout << settings_.vehicles[0].eventcameras.size();


it seems like a debug code. you can remove it

yun-long · 2021-03-15T16:35:53Z

flightlib/src/bridges/unity_bridge.cpp

+  //             << " seconds." << std::endl;
+  //   usleep((time_out / 10.0) * 1e6);
+  //   run_time += time_out / 10.0;
+  // }


I guess the code is commented out by accident.

yun-long · 2021-03-15T16:37:09Z

flightlib/include/flightlib/sensors/event_camera.hpp

+  float sigma_cp_;
+  float sigma_cm_;
+  int64_t refractory_period_ns_;
+  float log_eps_;


float or Scalar

yun-long · 2021-03-15T16:37:50Z

flightlib/src/bridges/unity_bridge.cpp

+  handleOutput(always);
+}
+
+bool UnityBridge::handleOutput(bool always) {


always is not used

yun-long · 2021-03-15T16:38:19Z

flightlib/src/bridges/unity_bridge.cpp

  // create new message object
  zmqpp::message msg;
+  logger_.info(sub_port_);


this seems like only for debug. remove it if not needed

yun-long · 2021-03-15T16:40:45Z

flightlib/src/sensors/event_camera.cpp

+                         std::end(events));
+  queue_mutex_.unlock();
+  std::string amount = std::to_string(event_queue_for_img.size());
+  logger_.warn(amount);


why need a warning here?

yun-long · 2021-03-15T16:42:54Z

flightlib/src/sensors/event_camera.cpp

+
+EventCamera::EventCamera()
+  : channels_(3), width_(720), height_(480), fov_{70.0} {
+  auto time = std::chrono::high_resolution_clock::now();


replace auto with std::chrono::high_resolution_clock::now

yun-long · 2021-03-15T16:45:25Z

flightlib/src/sensors/event_camera.cpp

+  }
+  logger_.error("empty events buffer");
+  return events;
+}


get which events? all events from the start or a single event at the current time step?

yun-long · 2021-03-15T16:48:13Z

flightlib/src/sensors/event_camera.cpp

+bool EventCamera::setCp(const float cp) {
+  if (cp <= 0.0) {
+    logger_.warn(
+      "The setting value for Eventcamera Cp is not valid, discard the "


what is Cp, maybe be specific here

yun-long · 2021-03-15T16:49:36Z

flightlib/include/flightlib/sensors/event_camera.hpp

+  float getCm(void) const;
+  float getCp(void) const;
+  float getsigmaCm(void) const;
+  float getsigmaCp(void) const;


are floats needed, or Scalar

yun-long · 2021-03-15T16:52:30Z

flightlib/src/bridges/unity_bridge.cpp

@@ -24,6 +24,7 @@ bool UnityBridge::initializeConnections() {
  pub_.bind(client_address_ + ":" + pub_port_);

  // create and bind a download_socket
+  logger_.info(sub_port_);


is this logger for debug only

yun-long · 2021-03-15T16:53:12Z

flightlib/src/bridges/unity_bridge.cpp

@@ -269,37 +322,100 @@ bool UnityBridge::handleOutput() {
        }
      }
    }
+    // feed events to the eventcamera
+    for (const auto& cam : settings_.vehicles[idx].eventcameras) {


try not use auto

yun-long · 2021-03-15T16:57:29Z

flightros/include/flightros/record.hpp

@@ -0,0 +1,90 @@
+#pragma once


the record.hpp can be more specific, e.g., "record_events.hpp"

but it is actually to record all kind of data, e.g. positions, images, ...

yun-long · 2021-03-15T16:59:33Z

flightros/include/flightros/testing.hpp

@@ -0,0 +1,94 @@
+#pragma once


is this testing.hpp file needed?

yun-long

some high-level comments:

use clang-format for automatically formating the c++ code
be consistent when using "float" or "double", I prefer using "Scalar"
delete codes that are not longer needed, such as std::cout , longer, or commented code
in flightros, put the ROS example into a single folder, e.g., a folder named "eventcamera"

gerni17 and others added 2 commits February 28, 2021 13:46

merged upstream and event camera works, normal camera not yet

5b8db66

antonilo requested a review from yun-long March 6, 2021 10:38

yun-long reviewed Mar 15, 2021

View reviewed changes

yun-long requested changes Mar 15, 2021

View reviewed changes

gerni17 added 3 commits March 27, 2021 23:14

added changes of comments

b9e2d97

fixed corner case of zero events

196c437

Merge branch 'master' into for_merge

eeb1eee

most changes for yulongs comments

3063be3

gerni17 changed the base branch from master to dev/eventcamera April 5, 2021 12:13

fixed merge conflicts

7ecdc58

yun-long merged commit 40cd22e into uzh-rpg:dev/eventcamera Apr 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Event camera #83

Event camera #83

gerni17 commented Feb 28, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

gerni17 Mar 26, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

gerni17 Mar 26, 2021

yun-long Mar 15, 2021

gerni17 Mar 26, 2021

yun-long Mar 15, 2021

gerni17 Mar 26, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

yun-long Mar 15, 2021

gerni17 Mar 26, 2021

yun-long Mar 15, 2021

yun-long left a comment

Event camera #83

Event camera #83

Conversation

gerni17 commented Feb 28, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yun-long left a comment

Choose a reason for hiding this comment