Juyang Weng

Career

Following his Ph.D., in 1990, Weng began his academic career as a visiting assistant research professor at the Beckman Institute of the University of Illinois, Urbana. From 1992 to 1998, he served as an assistant professor at Michigan State University, becoming associate professor in 1998 and professor in 2003. Retired now

Research

Weng's research revolves around grounded machine learning, spanning vision, audition, natural language understanding, planning, and real-time hardware implementations. He is also involved in technology transfer through his startup, GENISAMA, which focuses on grounded, emergent, natural, incremental, skull-closed, attentive, motivated, and abstract systems. His theoretical contributions include mathematically proving that Developmental Networks (DNs) he developed can learn any universal turing machines and establishing a theory on Autonomous Programming For General Purposes (APFGP), supporting Conscious Machine Learning.

Weng has worked on developmental networks from Cresceptron to DN3 to achieve the first-ever conscious learning algorithm which is free from "deep learning" misconduct. His research has been featured on Discovery Channel, Enel, and BBC.

Motion and structure analysis

From 1983 to 1989, Weng's research work during his master's degree and Ph.D. degree focused on the analysis of the motion of objects and estimating 3D structures from motion. He realized that such model-based approaches can provide with piecemeal insights but are too restrictive for understanding how animal brains learn vision and other brain skills. Soon after his Ph.D. degree work, he started Cresceptron.

Cresceptron

Cresceptron represented a direction that Weng later termed Autonomous Mental Development (AMD). In 1992, he and his collaborators pioneered the development of a framework titled Cresceptron for segmenting and recognizing real-world 3D objects from their images through automated learning. The framework was tested for visual recognition, specifically recognizing 3-D objects from 2-D images and segmenting them from cluttered backgrounds without the need for handcrafted 3D models. It employed techniques such as stochastic distortion modeling, view-based interpolation, and a combination of individual and class-based learning approaches. Cresceptron achieved seven significant accomplishments, including the development of techniques like learning large-scale 3D objects with a deep convolutional neural network (CNN) and feature-independent learning for extensive datasets, among others. It was also established that Cresceptron significantly differs from later "deep learning" networks due to its approach of developing a sole network using Hebbian learning (i.e., unsupervised in all hidden layers).

SHOSLIF

Weng introduced another framework named SHOSLIF which provided a unified theory and methodology for comprehensive sensor-actuator learning. It addressed single sensory problems as well as critical issues that Cresceptron faces, such as the automated selection of the most valuable features, the automatic organization of sensory and control information through a coarse-to-fine space partition tree, resulting in a remarkably low, logarithmic time complexity for content-based retrieval from extensive visual knowledge bases. It also deals with handling invariance through learning, enabling online incremental learning, and facilitating autonomous learning, among other objectives.

SAIL and Dav robots

From 1998 to 2010, Weng developed SAIL and Dav robots using sensory mapping models including self-aware self-effecting (SASE), staggered hierarchical mapping (SHM), and incremental hierarchical discriminant regression (IHDR) methods. It has been applied to the recognition of occluded objects, speech recognition, vision-guided navigation, and range-based collision avoidance.

Autonomous developmental networks

Since 2005, Weng and his team have been working on the development of brain-like and cortex-like Developmental Networks (DNs) and their embodiments Where-What Networks (WWNs) using brain-like architecture, including modeling pathways, laminar 6-layer cortex, and brain areas. In addition, they have analyzed how the brain deals with modulation, time, and space and have created three versions (DN1 through DN3) by 2023. A significant enhancement introduced in the transition from DN-2 to DN-3 involves initiating the brain-size network from a single-cell zygote. This means a fully autonomous process for brain patterning from a single cell. The key mechanisms of patterning include the Lobe Component Analysis (LCA) and Synaptic Maintenance, which automatically maintain the global smoothness of brain representation and local refinements of area representations. This approach enabled the developmental algorithm to progressively develop sensors, a complex brain, and motor functions in a sequential and self-organizing manner, ensuring that the wiring and pattern formation processes occur automatically from the initial conception stages throughout the entire life of the system.

These Developmental Networks (DNs) and Where-What Networks (WWNs 1–9) have been developed for versatile visual learning in complex environments. DNs can recognize objects and autonomously determine where and what to focus on using self-generated task context. Furthermore, these WWNs and DNs have been applied to general-purpose vision, temporal visual event recognition, vision-guided navigation, learning audition while learning to speak, and language acquisition as brain's responses to text temporal events.

Weng is the first to formally raise that robotic consciousness is necessary for AI, consciousness can and should be learned (i.e., developed), and proposed a fully implementable algorithm to do so. He proposed DN3 as the engine to conduct conscious learning where a robot is able to become increasingly conscious, like an infant and then a child, through its 'living' experience in the physical world which typically include human parents and teachers. However, there is no central controller within DN3's skull, emphasizing that consciousness should not be statically handcrafted and must encompass elements beyond a programmer's design.

Controversies

Since 2016, Weng has alleged instances of plagiarism and post-selection misconduct on a worldwide scale, but the implicated institutions have not yet acknowledged his allegations.

Plagiarism controversy

Weng alleged that many deep learning networks that use images of 3D objects copied their key idea from Cresceptron but almost all later deep learning publications did not cite Cresceptron. He highlighted that the Cresceptron (for 3D) is very different from the Neocognitron (for 2D) because the Cresceptron is a fundamental departure from Neocognitron. Cresceptron enables a neural network to grow incrementally from a zero-neuron hierarchy and learn 3D objects from their 2D images in cluttered scenes. This is different from the aspect graphs of the 1990s and all other methods that had an inside-skull human teacher as a central controller. This alleged plagiarism includes HMAX at MIT and the ACM Turing Award 2018. Without internal weight supervision like human manual selections and error-backprop, feature learning and sharing in hidden areas of Cresceptron are based on (unsupervised) Hebbian mechanisms.

Post-Selection controversy

Weng raised the issue of Post-Selection in AI and argued that it constitutes misconduct. He addressed that many AI methods require two steps in their training stage. The first step consists of training multiple systems by randomly fitting a fit data set. The second step consists of Post-Model Selection (Post-Selection). The Post-Selection chooses a few luckiest trained systems or relies on human manual parameter-tuning based on the systems’ errors on a validation data set. He alleged that Post-Selection in AI contains two types of misconduct: (1) cheating in the absence of a test, because the Post-Selection step belongs to the training stage; (2) hiding bad-looking data, because less lucky systems were not reported.

Weng further alleged that more categories of AI methods suffered from their Post-Selection steps, such as Neocognitron, HMAX, Deep Learning, Long Short-Term Memories, Extreme Learning Machines, Evolving Networks, Reservoir Computing, Transformers, Large Language Models, ChatGPT, and Bard, as long as they contain the Post-Selection step, which is either automatic or requires human manual tuning. He mathematically reasoned that the luckiest system on a validation set gives only an expected performance on a future test set that is only near the average performance of all trained systems on the validation set.

Weng has sued institutions to address the issue of alleged misconduct outside of academia including Alphabet, in the United States District Court for the Western District of Michigan (Civil Action No. 1:22-cv-998) and the US Court of Appeal 6th Circuit (Civil Action No, 23–1567).