Picture of Herman Bruyninckx

Prof. dr. ir. Herman Bruyninckx

Simple, though not easy…”

Full Professor (Gewoon Hoogleraar) at KU Leuven, and partime Full Professor at Eindhoven University of Technology.

herman.bruyninckx (@) kuleuven.be

Celestijnenlaan 300, bus 2420
B-3001 Heverlee (Leuven), Flanders, Belgium

Room 01.053. Tel: (+32) 16 32 80 56 (direct), (+32) 16 32 24 80 (secr)

ORCID identity: 0000-0003-3776-1025

KU Leuven logo
KU Leuven
Mechanical Engineering
(location)
How to reach Leuven.

TU/e logo
Eindhoven University of Technology
Mechanical Engineering
(location)

Research

I focus on integrating, into robots, as much formally represented domain knowledge as possible especially for realtime algorithms and other software close to the hardware, the controller(s) and the sensor(s). In other words, I am on a continuous quest for the holy grail of the (mythical) “robotics ontology”

This puts my research in the (currently sparsely populated) corner of Artificial Intelligence that is opposite to the (currently extremely popular) (deep) learning. My knowledge representations do include the relations with which the latter techniques can be integrated into any robotic system, in a systematic way. (Like any other AI technique for that matter.) That integration consists of how to configure the many “magic numbers” that the (so-called) “model-less” techniques require for a proper working in a specific task and application context; this configuration in itself requires quite some understanding of the intriciate dependencies between the perception, plans, control and monitoring activities in a robotic system.

My research takes place in close cooperation with Erwin Aertbeliën (super expert in dissecting and programming the most difficult robot tasks and in numerical solvers), Wilm Decré (my main liason with industrial projects), Joris De Schutter (former supervisor, co-creator of most of my “robot skills” R&D, reads robotics challenges as no-one else), Goele Pipeleers (for solvers of constrained optimization problems), Jan Swevers (main liason for all things control), René van de Molengraft (core co-creator of my approach towards system-of-systems architectures, and especially to “lazy” robot skills), Eric Demeester (main liason for shared control and industrial robot vision), Peter Slaets (main liason for unmanned (“autonomous”) shipping), Mark Versteyhe (main liason to the mechatronics industry in Flanders), and a manageably small set of PhD students and postdocs. All of the KU Leuven people have enjoyed the view from standing on the shoulders of our robotics founding father Rik van Brussel.

Research question 1

How can knowledge-driven (“affordance-based”) robot programming, perception and learning be made more realtime, while still taking into account more prior knowledge about the tasks, the robots, the objects they interact with, and the environment they have to survive in?

Preliminary answer: by exploiting Bayesian information theory to model all sensory-motor interactions, and to embed their runtime exploitation by means of model-predictive controllers and moving-horizon estimators.

Summary of results: the information and software architectures for the motion stack and the perception stack of robotics systems-of-systems are extremely similar, composable and formally verifiable. The “knowledge” is represented as constraints between parameters in the Bayesian model. The traditional Bayesian model is extended to be hierarchical, in the sense that it must be possible to let different sets of knowledge constraints apply to different parts of the Bayesian model; this is a pragmatic way to introduce the all-important concept of “context”.

Research driver 1

The societal expectation to have access to (only) trustworthy robots

Instead of chasing one of the many non-constructive “definitions” of levels of autonomy (like “Sheridan's 10”, Parasuraman, Sheridan & Wickens, IEEE Trans. Systems, Man, Cybernetics, 2000) we should design robots that can pass the “Trustworthy Turing Test” (TTT). That means that they are always able to answer the following questions, in increasing order of decision making quality/complexity and of what if? answerability:

Level Description
One system — One task
1 What am I doing?
2 Why am I doing it?
3 How am I doing it,…
4   …and how well am I doing it…
4b   …and how do I decide to stop doing it?
One system — Multiple tasks
5 What could I be doing instead,…
5b   …and still be useful,…
5c   …and how do I decide to switch what I am doing?
6 What is threatening my progress,…
6b   …and how can I make myself resilient,…
6c   …and how do I decide to add a particular resilience?
Multiple systems — Multiple tasks
7 What progress of others am I threatening,…
7b   …and how can I make myself behave better,…
7c   …and how do I decide to adapt a particular better behaviour?
8 What other machines and humans can I cooperate with,…
8a   …and how do I find out how can we coordinate our cooperation,…
8b   …and how do we decide, together, what coordination to adopt,…
8c   …and how do we monitor our coordination,…
8d   …and how do we decide that someone has cooperation problems?,…

The answers to these questions help human observers to assess whether the system is “aware” of the context, purpose and consequences of its actions, and the motivations behind its decisions.

The last time I looked, the state of the art in robotics was still at level 0…

Research driver 2

The societal expectation to have trustworthy Artificial Intelligence

These TTTs are tough to realise because it requires thorough scientific methods, but it is my hypothesis that they are the only way to build “AI” technology that is refutable/falsifiable, respects causality, and is predictable to the extend of being certifiable. In other words, robotic system developers can only claim to be working ethically if and only if they strive for full TTTs in all their systems.

Research question 2

What are the formal Domain Specific Languages (DSLs) that can make the knowledge representation (and hence the programming of robots) a lot more easy? And, at the same time, a lot more semantically consistent, and (hence!) deterministic, and (hence?) explainable, and (hence?) verifiable, and (hence!) certifiable, and (hence?) societally trustworthy.

Preliminary answer: by creating lots of small ontologies, with their Primitives, Relationships, Constraints and Tolerances encoded in languages such as JSON-LD, that support N-ary relationships and context-specific hierarchical composition as first-class citizens.

Summary of results: I was a key creator of the BRICS Component Model, and of its successor the System Composition Pattern, which is a scientific paradigm to support the design, development, deployment and runtime adaptation of complex robotics and other cyber-physical) systems.

Research question 3

Which new design paradigm can provide cheap, light and safe (hence, “lousy”) robot hardware? This is a necessary evolution before robotics platforms can become a commodity.

Preliminary answer: confidential, for now.

Research question 4

What is the essential and minimal structure to model the software aspects of robotic systems?
How should robot control software be developed in the future?
What architectural patterns can help us cope with the exploding complexity in knowledge, task variations, and distribution over several sub-systems?

Preliminary answers: (i) by systematically applying a small set of system-of-systems composition patterns, (ii) by clean separation of the information, software and hardware architectures, and (iii) by generating the robots' motions more and more by preview and precognitive control.

Preview control is the “information architectural” model behind Model-predictive Control (MPC) (and its estimation dual Moving Horizon Estimation); it adds a symbolic/modelling part to the numerical robot state, to represent the task-level aspects of intentions, progress and benefits of an ongoing robot action, to allow making decisions about altering that control on the basis of what the future is expected to bring. This information could be obtained as a side-effect of the control-level optimizations done in an MPC, by solving Constrained Optimization Problem using finite horizons over time and state space. The MPC state in itself can already be “hybrid”, in that it contains discrete as well as continuous parts; the symbolic part is involved in a “Constraint Satisfaction Problem” which is solved by reasoning systems and extends the MPC with a closed world of knowledge relationships (which is the symbolic equivalent of a “finite horizon”).

When the robot itself is able to fill in the symbolic information in the preview control model, we call this control mode pre-cognitive control. We're not there yet…

Deprecated answer: I started the Open RObot COntrol Software (OROCOS) project in 2001, with later “spin-off” projects Kinematics and Dynamics Library (KDL) and Bayesian Filtering Library (BFL). My first PhD student on this material, Peter Soetens, started his spin-off company Intermodalics in 2010, on the basis of his unique expertise with the Orocos code.

Research question 5

What is the essential and minimal structure to model the functional aspects of robotic systems, such that

  • all entities, relations, and constraints are given a unique and semantically unambiguous place.
  • no model must ever be changed when composed into a larger system, except for some configuration of parameters in the model.
  • all physical constraints can be covered: energy sources, power transformation to the mechanical domain, mechanical transmmissions, joints, kinematic chains.
  • all artificial constraints can be covered: tasks for individual robots as well as (cooperative) systems of robots.

Best-practice answer, after three decades of practicing, and just for the functional aspects:

Research questions “ICT in Society”

The following questions can not really be called “research questions”, because the answers are obvious. But, the last time I looked, these answers are not accessible in state of the art publications, and not at all in educational curricula. So, I publish the questions here, and leave the answers as a warming-up exercise to the reader.

Why don't highly-educated decision makers (yes rectors, CEOs and ministers, it's you I'm referring to…) understand that an ICT platform that is only accessible with a login creates huge ICT monopolies like Apple (AppStore/iTunes), Google (Play), Facebook, Twitter or LinkedIn? Aren't these the same people who do understand (I think…) why it does not make sense to let one login into the telephone system, the internet, or the World Wide Web.

Why do universities or government organisations put clickable logo's of the above-mentioned companies on all their websites, and hence introduce discrimination and stimulate inequality?

Why don't highly-educated decision makers understand the fallacies of the free market, and hence fail to create ICT regulation and guidelines that lead to free market ICT platform with fair entrance and competition conditions? What is so difficult about the golden rule that a provider of a platform (ICT and others) should never be allowed to become a provider of services on top of that platform? And why should decision makers never allow platform providers to put a password on the platform to access the data that actually are owned by the user?

Why don't they understand that all the investments they make to close the digital divide only makes it larger? For example, do they really believe that providing money to schools to use Microsoft Word/Excel, Apple FaceTime or the MathWorks Matlab helps the pupils to become empowered IT users in their later life? Maybe they don't realise that the moment these students leave school and start their career, their employers have to cough up several thousands of euros to let them continue with the same ICT habits?

Continuous education to individuals, organisations and companies

I consider an academic degree as a more than decent starting point for a professional career, but nothing more. Hence, I offer state of the art update classes in all areas of my expertise, at consultancy fees. Individuals, organisations (press, governmental administations,…) and companies can apply, starting from half a day to three day courses.

The fees go for 100% to KU Leuven or TU Eindhoven funds to support my research. Both universities have selected continuous education as one of its major missions, but have not yet been able to provide much in terms of concrete contents; so, mine is a humble contribution to that mission.

Projects

Current international projects in the European Union's H2020 Programme: Esrocos (H2020-ESA cooperation, 2016–2019), Ropod (2017–2019, coordinated by Nico Hübel), RobMoSys (2017–2020, coordinated by Enea Scioni).

Lessons learned from past projects: BRICS (Best Practice in Robotics, 2009–2013) and Rosetta (Robot control for skilled execution of tasks in natural interaction with humans; based on autonomy, cumulative knowledge and learning, 2009–2013) have helped me understand what step changes are required in the domains of, respectively, systems software engineering and task specification. In Pick-n-Pack (2012–2016), the insights gained in the above-mentioned projects were turned into innovative software solutions, in the context of “robotics” food production lines. RoboHow (2012–2016) complemented the above-mentioned ones by (preliminary versions of) formal representations of the knowledge of robot motions and tasks. Sherpa (2013–2017) and the ongoing H2020 projects allow the positively brutal confrontation of our insights with the real world of various challenging application domains, and with the strong but highly justified requirements from end-users and industrial integrators.

Publications

Contact me to get electronic copies of my publications.

I have been very active in promoting the introduction into the robotics domain of the separation of concerns concept, originally via the 4Cs (of Radestock and Eisenbach, 1996, see below), which I refined into the 5Cs: Computation, Communication, Coordination, Composition, and Configuration.

The first “real” publication about the 5Cs was this White Paper, created in the context of the Robot Standards (RoSta) project: Erwin Prassler, Herman Bruyninckx, Klas Nilsson, and Azamat Shakhimardanov, The Use of Reuse for Designing and Manufacturing Robots. Klas deserves the credit of introducing me to the seminal paper by Matthias Radestock and Susan Eisenbach, Coordination in evolving systems, Trends in Distributed Systems. CORBA and Beyond, Springer-Verlag, 1996, pp. 162-176.

A more complete paper on the 5Cs, and on how to use them not just for separation but also for constructive composition has been published in JOSER: The 5C-based architectural Composition Pattern: lessons learned from re-developing the iTaSC framework for constraint-based robot programming, Dominick Vanthienen, Markus Klotzbuecher, Herman Bruyninckx, Vol. 5, No. 1, 2014.

The documentation of Markus Klotzbücher's rFSM package on Github can be found here.

Teaching & Education

The contributions to the education of our young engineers that I value most are my emphasis on (i) system-level thinking, and (ii) attitude of constructively critical evaluation of all available sources of information, starting with pseudo-peer reviewed open content such as the Wikipedia. Our students typically score poorly on both aspects, which I think are fundamental for Europe's ability to maintain an innovative R&D ecosystem. The future does not belong to those who posess the most knowledge, but to those who are able to understand how and where to apply that knowledge.

My most “revolutionary” contribution to education is to use professional mailing lists as first-class teaching tool: this is the most effective (albeit labour intensive and not always efficient…) approach to provide learning feedback to students on an individual basis, answering to their problems when they are ready for it. This best practice comes directly from my long-term, intensive immersion in, and contributions to, the “open source” community.

Universities recently started to promote asocial media technologies as revolutionary additions to traditional educational practice. I'm sorry, but that technology existed (and has been used, by myself and many others) already a long time before companies like Facebook, Twitter and Balckboard made that technology proprietary and put it behind a passwords.

People

Senior research staff.
The following researchers form the core of robotics team: Erwin Aertbeliën, Wilm Decré, Enea Scioni, Johan Philips, Nico Hübel, and Zhang Lin.

PhD students

PhD alumni

Working with me

If you want to come and work with me, I expect you to be a full-time user of Linux, text-based editors (such as Vim or Emacs), text-based email with inline posting, Blender, version control systems (e.g., git), LaTex and HTML5 and SVG, (for documents as well as present/ations). Contributions to Free and Open Source Software projects are very much stimulated, and contributions to “a friendly Wikipedia page in your neighbourhood” should become part of your daily hygiene.

The most important thing I can offer to potential post docs is a lot of opportunities to get immersed into the most vibrant core of the Dutch-Flemish robotics research scene, academic as well as industrial, including lots of interactions with dozens of robotics groups in Europe.

I am a firm believer in the maturity and responsibility of master and PhD students. Hence, I do not want to be their “supervisor” but rather their “somewhat more experienced coach”. In return, I expect them always to have a clear idea about where exactly they want to go with their research. My rule of thumb for a PhD student is to have 2–3 research hypotheses written out in full, at all times. They need them, not only to explain to visitors what their research is all about, but also to keep their strength and self-confidence, since I flood them continuously with (only potentially) good ideas, papers and software, with constructive criticism, and with stimuli to “think weird” and “design big”. I do realise that such a turmoil of scientific discussions and doubts can take some time to adapt to, and requires strong nerves to keep one's research focus. However, I do not apologize for this behaviour of mine.

Student internships and master theses

I welcome Master students from universities all over Europe, and I'm especially interested in computer-literate students (Linux, C, Lua, HTML5, and, to a lesser extent, “everything and the kitchen sink” wrong-level languages such C++ or Java). I will stimulate them to contribute to Free and Open Source Software projects, to make the latter better suited for robotics.

Curriculum

From 2008 to 2015, I have been leading the robotics community in Europe, first as Coordinator of the seminal network EURON, and in 2013–2015 as Vice-President Research of the euRobotics association.

Doctor honoris causa (“æresdoktor”) of the University of Southern Denmark (Syddansk Universitet, Odense, Denmark), on October 3, 2014.

From its inception in 2000 until 2017, I have been a member of the Jury for the (then “EURON” now “euRobotics”) Georges Giralt PhD Award. From 2008 till 2014, I acted as the Chairman of this Jury.

Professor at KU Leuven since October 2008.
Associate Professor at KU Leuven since October 2003.
July–August 2002: visiting the Centre for Autonomous Systems at the Royal Institute of Technology in Stockholm, Sweden, with Prof. Henrik Christensen.
April–August 1999: sabatical with the Robotics group at Stanford University, with Prof. Oussama Khatib.
Assistant Professor at KU Leuven since October 1998.
Mar. 1996–Aug. 1996: Postdoc at GRASP Lab, University of Pennsylvania, Philadelphia, U.S.A., with Vijay Kumar.
1995–2003: Postdoctoral Fellow of the Fund for Scientific Research (FWO) in Flanders.
1989–1995: Research Assistant at the University of Leuven, Department of Mechanical Engineering.
PhD (1995), Kinematic Models for Robot Compliant Motion with Identification of Uncertainties, under supervision of Joris De Schutter. [abstract] [A4 paper, gzipped PostScript]
Military service (1988–1989).
Master-after-Master Mechatronics (1988).
Master (“Burgerlijk Ingenieur”) Computer Science (1987).
Master (“Licentiate”) Mathematics (1984).
Deployed to the world: 22 December 1962, Turnhout, Belgium.

Miscellaneous

I do not have a Skype account, nor do I take part in asocial media. I am not planning to get those accounts, because of pragmatic and ethical principles: these initiatives introduce proprietary protocols and/or prevent bias-free, inter-community, multi-vendor communication. The inevitable result is to create monopolies, and hence prevent fair markets of VOIP or social networking as emerging communication instruments. It's only 40 years ago that our society succeeded to escape from the traditional telecom monopolies, but it seems not to have learned anything from those experiences…

If you are prepared to invest in fairness and freedom, I suggest to use the facilities offered by the free market of teleconferencing via the traditional telephone line and to use open VOIP protocols. Or, preferably, use Open Standards formats, such as WebRTC, and user-friendly implementations of it such as JitsiMeet'.

I am strongly convinced of the long-term advantages of using only Open Standards in all ICT matters: vendor independence, software independence, better chances of long-term archiving, stimulation of better decoupled ICT solutions, etc. So, please, send me only plain text, HTML, PDF, or ODF messages and documents in your electronic communication.