# Fileset

[2023_Alon_Diaz-Alvarez_Sci.Adv._sciadv.adg3289.pdf](https://mdr.nims.go.jp/filesets/5cf71b20-2592-4357-ac93-1e064997a2bb/download)

## Creator

Alon Loeffler, Adrian Diaz-Alvarez, Ruomin Zhu, Natesh Ganesh, James M. Shine, [Tomonobu Nakayama](https://orcid.org/0000-0001-9696-475X), Zdenka Kuncic

## Rights

[Creative Commons BY Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/)

## Other metadata

[Neuromorphic learning, working memory, and metaplasticity in nanowire networks](https://mdr.nims.go.jp/datasets/3df003da-b82f-46be-898f-6b39b7e0f933)

## Fulltext

Neuromorphic learning, working memory, and metaplasticity in nanowire networksPHYS ICSNeuromorphic learning, working memory, andmetaplasticity in nanowire networksAlon Loeffler1*†, Adrian Diaz-Alvarez2,3*†, Ruomin Zhu1, Natesh Ganesh4,5, James M. Shine1,6,7,Tomonobu Nakayama1,3,8, Zdenka Kuncic1,3,9*Nanowire networks (NWNs)mimic the brain’s neurosynaptic connectivity and emergent dynamics. Consequent-ly, NWNsmay also emulate the synaptic processes that enable higher-order cognitive functions such as learningandmemory. A quintessential cognitive task used tomeasure humanworkingmemory is the n-back task. In thisstudy, task variations inspired by the n-back task are implemented in a NWN device, and external feedback isapplied to emulate brain-like supervised and reinforcement learning. NWNs are found to retain information inworkingmemory to at least n = 7 steps back, remarkably similar to the originally proposed “seven plus or minustwo” rule for human subjects. Simulations elucidate how synapse-like NWN junction plasticity depends on pre-vious synaptic modifications, analogous to “synaptic metaplasticity” in the brain, and how memory is consol-idated via strengthening and pruning of synaptic conductance pathways.Copyright © 2023 TheAuthors, somerights reserved;exclusive licenseeAmerican Associationfor the Advancementof Science. No claim tooriginal U.S. GovernmentWorks. Distributedunder a CreativeCommons AttributionNonCommercialLicense 4.0 (CC BY-NC).INTRODUCTIONThe brain’s powerful information processing capacity can be largelyattributed to neuronal microcircuits established by synaptic connec-tivity patterns (1, 2). Precisely how neurosynaptic connectivity givesrise to higher-order cognitive functions such as learning andmemory remains elusive (3). However, an important clue is thatneural connectivity is spatiotemporally sparse and dynamic (4, 5).Here, learning and memory are demonstrated in a unique physicalsubstrate with these properties.Nanowire networks (NWNs) emulate the physical nature ofneurons and synapses in the brain (6). They are “neuromorphic”by virtue of not only their efficient integration of processing andmemory in nanowire-nanowire cross-point junctions (7, 8) butalso their ability to mimic both threshold-driven spike-like neuro-nal dynamics and conductance-based synapses (9, 10). Nanowirejunctions exhibit resistive memory (“memristive”) switchingbetween high and low resistance states (11). Because of NWNself-assembly, these memristive junctions are interconnected in aheterogeneous circuitry with recurrent feedback loops (12). Thus,NWN devices operate in a fundamentally different way from top-down fabricated memristor devices in a cross-bar architecture (8).In particular, NWNs exhibit emergent nonlinear dynamics as aresult of the interplay between their memristive junctions and het-erogeneous, recurrent network connectivity (13–15).Previous studies have demonstrated how nonlinear dynamicscan be harnessed for learning by treating the NWN as a physical“reservoir” in a reservoir computing paradigm [e.g., (9, 13, 15–23)]. This paradigm exploits the network’s ability to nonlinearlytransform dynamical input signals into a higher-dimensionalfeature space, such that the outputs are linearly separable (24–26).NWN device readouts can then be used in a highly computationallyefficient linear output layer, where only linear weights need to betrained to complete a desired machine learning task (27). In con-trast, learning in the brain is thought to occur via three main mech-anisms (28): supervised learning, typically linked to the cerebellum(29–32); reinforcement learning (33), typically linked to the basalganglia (33–37); and unsupervised learning, typically linked to thecerebral cortex (28). In our recent study (38), we demonstratedHebbian-like unsupervised learning via signal transduction path-ways in NWNs. We reshaped these conductance pathways by alter-ing the spatial location of input and output electrodes, as well as theorder in which they were activated.Such “dynamic pathway tuning” revealed that NWNs preserveinformation from previously established pathways when formingnew pathways through the network, analogous to how synaptic plas-ticity in the brain depends on previous synaptic modifications (39).Here, we investigate the other two mechanisms of learning, whichare more context dependent. Supervised learning encapsulates aniterative process whereby the system’s response to a given input isevaluated against a desired outcome, and deviations from thatoutcome are used to adjust adaptive elements within thesystem (40).In reinforcement learning, synaptic weights are modified in re-sponse to information related to positive (or negative) feedback(33). Here, these brain-inspired learning mechanisms are physicallyimplemented in NWNs, extending previous studies (38, 41) by ex-plicitly applying context-dependent external feedback.In addition to demonstrating brain-like learning in NWNs, wealso demonstrate working memory (WM) by implementing se-quence memory tasks inspired by the well-known cognitive task,the n-back task (42–45). In experiments with human subjects, then-back WM task requires participants to identify whether eachstimulus (e.g., visual pattern) in a sequence matches a stimulusthat was presented n-steps back (43). As n increases, reactiontimes tend to increase, and accuracy tends to decrease due to1The University of Sydney, School of Physics, Sydney, Australia. 2InternationalCenter for Young Scientist (ICYS), National Institute for Materials Science (NIMS),Tsukuba, Japan. 3International Center for Materials Nanoarchitectonics (WPI-MANA), National Institute for Materials Science (NIMS), Tsukuba, Japan.4National Institute of Standards and Technology (NIST), Boulder, CO, USA.5University of Colorado, Boulder, CO, USA. 6Brain and Mind Centre, The Universityof Sydney, Sydney, Australia. 7The University of Sydney, School of Medical Sciences,Sydney, Australia. 8Graduate School of Pure and Applied Sciences, University ofTsukuba, Tsukuba, Japan. 9The University of Sydney Nano Institute, Sydney, Aus-tralia.*Corresponding author. Email: aloe8475@sydney.edu.au (A.L.); adrianlocdiaz@gmail.com (A.D.-A.); zdenka.kuncic@sydney.edu.au (Z.K.)†These authors contributed equally to this work.Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 1 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023http://crossmark.crossref.org/dialog/?doi=10.1126%2Fsciadv.adg3289&domain=pdf&date_stamp=2023-04-21processing load (44, 46). Furthermore, regions of the brain relatedto verbal WM processes tend to show increasing magnitudes of ac-tivation during large n values (46). WM is thought to pertain toshort-term memory and involve information manipulation (47,48). The ability to temporarily hold and manipulate information re-quires adaptive processing of multiple incoming dynamical inputswhile retaining information about previously encoded input. Thismeans that synaptic connections that form memories must be pro-tected from being overwritten when storing new information(49, 50).Through sequence memory tasks inspired by the n-back WMtask, we demonstrate the ability of NWNs to recall previous infor-mation while continually processing new information. In addition,we show how information initially in short-term WM may be con-solidated into long-term memory through physical reinforcementlearning (PRL), which manipulates topological reconfiguration ofNWNs via pathway strengthening and pruning.RESULTSFigure 1 presents an overview of the NWN device (both physicaland simulated) and the setup designed to implement supervisedlearning and PRL in tasks inspired by the n-back protocol. Theaverage network density for the physical system shown in Fig. 1Ais ≈0.76 nanowires/μm2, similar to that used in the study byDiaz-Alvarez et al. (14). The simulated network had 698 nanowirenodes and 2582 edge junctions (0.12 nanowires/μm2). See Methodsfor full details of the physical and simulated NWN device.Figure 1B summarizes one sample epoch, consisting of trainingand testing strategies developed to demonstrate the n-back protocolusing the device. During training, the n-back protocol involves theNWN receiving n-1 unique nontarget samples after receiving atarget pattern cue. The NWN is trained to recognize the targetcue via a supervised learning strategy, which nudges a selectedoutput toward a fixed current threshold θ, with all other outputsnudged away from θ. The nudging protocol occurs via a gradientdescent-like method (see Eq. 2 in Methods), using the discrepancybetween actual and desired output currents. While supervisedlearning is implemented during training, PRL is instead implement-ed after testing recall of the target cue, when the value of θ can bemodified to provide feedback to the NWN. To test the network’sWM, θ is kept unmodified, corresponding to no PRL implemented.With PRL, θ is changed ahead of the next epoch based on the net-work’s performance: Target outputs receive more current, and non-targets receive less current. This tests the network’s consolidation ofitems initially held in short-term WM into long-term memory. Insummary, supervised learning nudges the output currents toward θduring training, while PRL controls the value of the current thresh-old θ after testing. See Methods for full details.Task 1: Physical binary classificationFigure 2 compares results for classification of two 2 × 2 patterns (i.e.,four inputs to the network) for n = 2 without and with reinforce-ment (see full task 1 description in Methods, including Algorithm1). Both experimental (Fig. 2A) and simulation (Fig. 2B) resultsdemonstrate that once PRL is introduced, it markedly improvesbinary classification accuracy under the n = 2 protocol (i.e.,testing after two training samples of the nontarget pattern are pre-sented to the network). In experiment, 44 of 50 epochs achieve anaccuracy of 100% with PRL, compared to only 23 of 50 without.Similarly, in simulation, 32 of 40 epochs are 100% accurate withPRL and 23 of 40 without. The worse performance without PRLresults from the network successfully training and recalling theprimary conduction pathway for one target drain, but not theother. This is most evident in the experimental results. Insetpanels in Fig. 2 show that supervised learning increases currenttoward the respective threshold (θ1 or θ2) of the target drain as aconsequence of voltage adjustments on the corresponding electrode(see fig. S1 for drain voltages).Simulated network connectivity maps (Fig. 2B) qualitativelyshow the conductance pathways that form between source anddrain electrodes for each pattern. Quantitatively, memristive junc-tions along the pathways experience a conductance gain under PRL,whereas without reinforcement, junctions can decay and reset (seefig. S2).Task 2: Complex binary classificationFigure 3 shows results for binary classification of 3 × 3 patterns(nine inputs) of either “+” or “x” (5-bits). For this task, thenumber of samples, n-1, between the first training sample and thetesting sample was increased. This meant that n-back increased as n= 2,3,4,5, or 6 (see Fig. 3A and task 2 description in Methods). Oth-erwise, task setup and implementation were similar to task 1.Both experimental and simulation results in Fig. 3D show amarked improvement in classification accuracy with reinforcementcompared to without reinforcement. In experiment, accuracy in-creases from 0.48 to 0.71 for n = 2, and the maximum increase isfrom 0.41 to 1.00 for n = 6. Similarly, in simulation, accuracy in-creases from 0.71 to 0.93 for n = 2, and the maximum increase isfrom 0.53 to 0.85 for n = 6. In experiment, mean accuracy rangesfrom 0.71 to 1.00 when reinforcement is applied, while without re-inforcement, it is similar to chance levels (mean accuracy rangesfrom 0.41 to 0.56). In contrast, simulation results with reinforce-ment show a narrower accuracy range (0.93 to 0.85), and simulationresults without reinforcement show a steady decline with n (0.71 to0.53). This is not observed in the experimental results, which show amarked increase in accuracy under reinforcement for n > 3.These results can be understood in terms of the memristiveswitching junctions responsible for the conductance pathwaysthat represent the physical manifestation of binary classificationby the network. During training, the NWN establishes input-output conductance paths for each of the two patterns (Fig. 3C).As n increases, the NWN receives more training samples from non-target patterns than the target. When this occurs without reinforce-ment, the memristive junctions corresponding to the target patterndecay, so that when the target pattern is next presented, the decayingconductive nanofilaments must reform. In experiment, a 3-hourwait time between each n is implemented (see Methods);however, it is difficult to determine to what extent the conductivepathways have decayed. In task 2 simulations, all junctions werecompletely reset between each n trial, so the accuracy without rein-forcement decreases consistently with n. The higher accuracy for n= 2 to 3 compared to the experimental results can be attributed tothe assumption made in the model that all junctions decay at thesame rate (see fig. S4 for additional results on varying the junctiondecay parameter). With reinforcement, the conductance pathwaysare prevented from decaying, so the NWN is able to recall thetarget pattern, even for large n. The increase in accuracy with nLoeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 2 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023evident in the experimental results may be attributed to nanofila-ments forming faster than they decay. This is not observed in thesimulation results due to the model assumptions mentionedabove. This reinforcement effect is similar to the notion of popula-tion coding in the mammalian brain (51), although it is difficult todisentangle innate from learned features of classification in in vivobiological brains (52). This highlights the importance of using non-biological, physical neural networks to understand how informationis processed in networks like the brain.Task 3: Working memoryFigure 4 (A and B) show a schematic of the WM n-back task, wherethe NWN receives varying nonrepeat sample 3 × 3 patterns bothbefore and during the n-back sequence (see task 3 description inMethods, including Algorithm 2). During training, each pattern ispresented only once and in random order. Unlike the previoustasks, therefore, no repeat training is possible during a single train-ing-testing epoch, and n varies from epoch to epoch.Figure 4C shows the experimental and simulation recall accuracyresults, sorted by n, while Fig. 4D shows the corresponding resultsplotted per epoch. Both the experimental and simulation resultsshow that themean accuracy in recalling the target pattern improvesmarkedly under reinforcement compared to no reinforcement.Without reinforcement, the experimental accuracy declines steadilywith n from 0.61 (n = 1) to 0.37 (n = 7), while the simulation accu-racy declines more steeply from 0.95 (n = 1) to below chance accu-racy (1/7 ≈ 0.14 odds of correctly selecting the target patternat random).Although a contributing factor may be differences between sim-ulation and experimental time scales (as the experimental NWN hasmany more wires and junctions and a 3-hour rest period was in-cluded between each trial), the difference between experimentaland simulation results without reinforcement may be primarily at-tributed to differences in junction decay rates. Specifically, the sim-ulation model assumes that all junctions decay at the same rate,whereas experimental NWNs are heterogeneous, with a range of fil-ament formation and decay rates due to varying nanowire thick-nesses and stochastic effects in the nanoscale junctions. Thesimulation results reveal that, with no reinforcement, recall ishighly sensitive to junction decay rates; both recall accuracy andmaximum n-back values decline with faster decay (see fig. S5),i.e., as previously established pathways are more quickly forgotten.This is analogous to metaplasticity in the brain, which describes thedependence of synaptic plasticity on the history of synaptic modifi-cations (39) .Without reinforcement, the n-back task is a measure of WM.With reinforcement, both experimental and simulation results inFig. 4 (C and D) show a high recall accuracy is maintained for alln. This indicates that short-term WM has changed to a longer-term memory that is independent of forgetting associated withjunction decay. The mechanism responsible for this memory con-solidation under PRL is explored next.Network connectivityThe network connectivity snapshots presented in Fig. 5 reveal thesimulated NWN states during early (t = 14 s) and middle (t = 558 s)stages of testing target pattern recall from WM during the n-backtask (for n = 3). The corresponding input pattern is shown on theleft of Fig. 5 (A and C), with zeros (purple) and ones (green) corre-sponding to inactive and active sources, respectively. During thefirst testing epoch (t = 14 s), the connectivity maps (Fig. 5, A andC) and Gj histograms (Fig. 5F) with and without reinforcement areidentical. This is because conductance paths are not yet conditionedby PRL. By the later testing epoch (t = 558 s), however, the effect ofreinforcement is noticeable, with a strong conductance path to thetarget drain now evident (Fig. 5D) and weaker paths to nontargetdrains. This occurs because the target drain is reinforced by increas-ing the current output threshold, while all other nontarget drainsare “punished” with lower current thresholds. Consequently, con-ductance pathways to the target drain are strengthened and remem-bered by the network, while pathways to nontargets are pruned andforgotten.The effect of reinforcement is visualized in Fig. 5E, where redpaths represent pathways strengthened by reinforcement (i.e.,higher Gj) over time, and blue paths represent pruned pathways.The difference ΔGj is calculated by first subtracting the maps att = 14 s and t = 558 s [i.e., Fig. 5 (A and B) without reinforcementFig. 1. Overview of NWN device and setup for supervised learning, PRL, andthe n-back task protocol. (A) Left: Optical micrograph showing physical silvernanowires dropcast onto a substrate between 9 × 9 contact electrodes (nineinputs and nine outputs; inset). Scale bar, 10 μm. Right: Schematic of simulatedNWN device with four input electrodes and four output electrodes, showing con-ductance pathways from source to drain electrodes (green and red, respectively).(B) Schematic of one epoch (training + testing) of an n-back task protocol. Training:Summary of training protocol in which presentation of target pattern A is followedby n-1 interference (nontarget) patterns. Schematic illustration of the NWN multi-electrode device setup demonstrating the training protocol. During training, adigital pattern (A) is cued to analog electrodes and fed into the device inputs;outputs are recorded and compared with the target, after which drain currentsare adjusted, nudging target output currents closer toward a fixed current thresh-old θ via supervised learning, and nontarget outputs away from θ. Testing:Summary of testing protocol in which the target pattern A is presented after ntraining patterns, and device outputs are compared with the target before thenext epoch starts. After testing: If the outputs do not match the target and PRLis not applied, then the epoch fails; if this occurs with PRL, then θ is increasedfor the target and decreased for the nontargets for the next epoch.Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 3 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023Fig. 2. Binary classification of 2 × 2 patterns with n = 2 (task 1). Top: (A) Experimental setup schematic of training a NWN for two unique 2 × 2 patterns. Greenelectrodes represent active sources (inputs), purple electrodes are inactive, and red electrodes are active drains (outputs; D1 = drain 1, D2 = drain 2). Patterns can bepresented in two possible orders (order 1 or 2), for each of which the target pattern is different. Bottom left: Without reinforcement: (B) Experimental results. Draincurrents (solid blue and red lines, left axis) and classification accuracy (blue and red dots, right axis) versus time. Horizontal dashed lines represent training thresholdθ1 for drain 1 (blue) and θ2 for drain 2 (red). Inset shows close-up of drain current over two training and testing (green shade) epochs in Δt = 35 to 43 s. During the firsttesting period (t = 35 to 38 s for no reinforcement and t = 64 to 65 s for reinforcement), order 1 is presented to the network. During the second testing period, order 2 ispresented to the network. (C) Simulation results. Inset shows close-up of drain current over two epochs in Δt = 48 to 60 s. Simulated network visualization snapshots(nodes = nanowires and edges = junctions) showing junction conductance states (Gj, colorbar) during testing at t = 53.0 (drain 2 target) and t = 59.0 (drain 1 target). Activeand inactive source nodes are in green and purple, respectively, with active drain nodes in red and target drain labeled. Bottom right: Same as left but with reinforcement.Insets show zoom-in over epochs in (B) Δt = 64 to 68 s and (C) Δt = 48 to 60 s.Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 4 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023and Fig. 5 (C and D) with reinforcement] and then subtracting non-reinforced paths from reinforced paths. What remains is a topolog-ical reconfiguration map, highlighting the paths strengthened andpruned by reinforcement. Pathway strengthening occurs inregions closer to the target drain, while pruned pathways are in non-target regions. See fig. S6 for intermediate connectivity maps.Differences in pathway strengths are quantified by Gj histogramsin Fig. 5F, which are identical during early testing, while during latetesting, more junctions exhibit higher Gj values as a result of rein-forcement. For these simulation results, a stronger filament decayparameter (b = 2) was used to enhance the visual contrast of con-ductance pathways in the functional connectivity maps in Fig. 5, butthis does not change the fundamental nature of pathway selectivitythat PRL produces. Other connectivity maps with varying b valuesare presented in fig. S7. The full videos from which these snapshotswere taken are also available in the Supplementary Materials.These findings highlight two unique memory capabilities inNWNs. First, the supervised learning paradigm without PRLreveals the importance of the history of junction changes to WM.The second memory capability is that of memory consolidationand occurs with the help of PRL. By manipulating the currentthreshold θ, specific pathways are reinforced with substantiallygreater current output, while other pathways are suppressed. Thisis realized as long-term memory for the strengthened pathways.Simulation results suggest that this activity is akin to memory con-solidation via synaptic strengthening (1).DISCUSSIONThis study is the first to demonstrate a nontrivial cognitive task—inspired by the WM n-back task—in a physical non–CMOS (com-plementary metal-oxide semiconductor) substrate with nativeFig. 3. Binary classification of 3 × 3 patterns for varying n (n ≥ 2) with and without physical reinforcement (task 2). (A) Experimental schematic of task 2 n-backvariation for two unique 3 × 3 patterns, “x” (pattern A) and “+” (pattern B), respectively. In epoch 1, x is presented during training as the target, followed by two (if n = 3)interference patterns +, and then x is presented again during testing. In epoch 2, the opposite order occurs. (B) Input pattern (as 1D vector) and corresponding inputnodes of graphical network representation used in simulation with two drain nodes (target in red, nontarget in purple). (C) Same as Fig. 2A but for five voltage inputs(green) corresponding to 5-bit patterns x (left) and + (right). (D) Experimental (navy) and simulation (orange) results with (solid lines) and without (dashed lines) rein-forcement. Chance accuracy (teal) is shown for comparison. Shaded areas represent SEM across epochs.Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 5 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023neuromorphic properties (i.e., not requiring implementation ofneuromorphic algorithms).In a previous study, Neftci and colleagues (53) demonstrated asimple cognitive task by emulating spiking neurons in a CMOSsystem. Their method used an intermediate computational layerin which silicon neurons are configured as soft winner-take-all(WTA) networks (54). The WTA mechanism has been reportedin previous NWN studies (14, 15, 38, 41, 55, 56). Functional connec-tivity maps in the current study, generated by the simulations, indi-cate that the network uses more than one key pathway, in contrast toprevious findings. This is because of the low voltages used in thisstudy, which are well below the threshold needed to activate theWTA path. This is to ensure that the network is maintained in anintermediate conductance state, enabling control of conductancepaths via the electrodes. We previously visualized conductancepathway formation in a similar multielectrode NWN device usinglock-in thermography (38). Despite the poor spatial resolution, wewere able to demonstrate the principle of reshaping conductancepaths in the network by dynamically changing the spatiotemporalpatterns of input signals delivered by the electrodes. In that study,we used Ag@TiO2 nanowires as Ag–polyvinyl pyrrolidone (PVP)nanowires, used in this study, are difficult to image using this tech-nique due to their much lower resistance, making them more sus-ceptible to damage by Joule heating.The training methods introduced here for learning a cognitivetask have strong links to two unique neuroscientific learning theo-ries. The first method, in which “nudging” was used, is similar tosupervised learning in the brain (29). This method is also similarto the gradient nudging described in Æqprop (57) or other inmaterio gradient descent methods such as described by Boon andcolleagues (58). Diaz-Alvarez et al. (41) previously demonstrated as-sociative routing in an Ag-PVP NWN using the same multielec-trode device configuration as used in this study. They effectivelytrained pathways by opening and closing selected electrodes toprompt the network to use specific pathways and associate themwith specific spatiotemporal patterns delivered by the electrodes.However, they found that this technique was unable to maintain re-liable pathway selectivity, particularly as more paths became estab-lished, which limited the ability to train multiple different patterns.By implementing selective feedback (PRL), our study demonstrateshow the strengthening and pruning mechanism underlying PRLcan control specific unique pathways to enable training of multipledistinct patterns and long-term memory of a target pattern.When supervised learning is implemented in NWNs withoutany reinforcement, drain electrode voltages are altered andnudged closer to the target. However, because of the finite decayrate of NWN junctions (59), coupled with a fixed current threshold(θ), the conductance pathways are only remembered temporarily,reflecting the network’s WM capacity. In humans, WM is anFig. 4. WMmultipattern n-back task (task 3). (A) Experimental schematic of task 3 n-back variation for seven unique 3 × 3 patterns. Pattern A is always selected as thetarget; however, its location is semirandomly varied, to change n. The order of the interference patterns (B to G) is random. (B) Same as Fig. 3C but for three voltagesources (green) corresponding to target pattern (A shown) and seven output electrodes (one target drain, red). (C) Mean recall accuracy in experiment (navy) and sim-ulation (orange), with and without reinforcement, for varying n (sorted by n). Shaded regions represent SEM. Chance accuracy (teal) represents a one-in-seven chance ofcorrectly classifying the target pattern over the six alternative patterns. (D) Mean accuracy sorted by epoch and corresponding θ threshold values;N is the number of trials.Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 6 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023example of information retention and consistent manipulation viasynaptic modifications until the information is no longer needed(48), at which point it decays in seconds up to minutes or isencoded (60).The cognitive task used in this study, a sequence memory taskinspired by the n-back task, is extensively used in cognitive psychol-ogy for testing WM in humans (42–44, 61). Sequence memory andn-back memory tasks have also been applied to recurrent neuralnetworks with bio-inspired topology (62). Well-known studies inhumans originally suggested a capacity to store 7 ± 2 items inWM (63), although subsequent studies estimate it at closer tothree to five “chunks” of memory (64, 65). Here, the n-back taskwas adapted into subtasks that could be implemented in NWNs.Task 3, the most similar to the original n-back task (61), showedthat NWNs can store up to seven items in memory (and potentiallymore) at substantially higher than chance levels without reinforce-ment training and near-perfect accuracy with reinforcement train-ing. One theory ofWM at the synaptic level describes how an item ismaintained in WM via increased residual calcium levels at presyn-aptic terminals of the neurons that code for that item (66). SinceFig. 5. Simulated NWN connectivity snapshots duringmemory recall. (A and B) Network connectivity maps visualizing junction conductance (Gj) snapshots at early (t= 14 s) and middle (t = 558 s) testing periods of the WM n-back task (with n = 3), respectively, without reinforcement. Active and inactive input electrodes are highlightedin green and purple, respectively, with active drain electrodes in red and target drain indicated. (C and D) Same as (A) and (B) but with reinforcement. (E) Topologicalreconfiguration map highlighting the junction conductance change ΔGj between reinforced and nonreinforced paths. Values are calculated by comparing connectivitymaps as follows: (d − c) − (b − a). (F) Gj histograms corresponding to (A) and (C) (top) and (B) and (D) (bottom). For clarity, Gj is thresholded at 2 × 10−5 S.Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 7 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023removal of residual calcium is a relatively slow process (around 1 sin humans), memories can be held over this time without the needfor further spiking (67). The depletion of residual calcium is con-ceptually similar to atomic filament decay in NWNs (6).The second method of learning implemented in this study, PRL,is similar to reinforcement in the brain, which is thought to occur, atleast in part, via strengthening of synaptic dopamine channelsthrough Hebbian plasticity, in response to a positive (or negative)outcome (35, 68). Contrastingly, and particularly during early de-velopment, when a synaptic pathway is unused, unwanted, or pun-ished, it is pruned (69). Pruning occurs via a weight-dependentsynaptic modification process called neuronal regulation (70).This study showed both reinforcement of desired pathways viaPRL, as well as pruning of penalized pathways in NWNs.A clear distinction must be made between non-PRL results andPRL results in the n-back task. Without PRL, task performance re-flects the network’s WM capacity, i.e., its ability to temporarilyrecall information pathways while establishing new ones. WhenPRL is introduced, however, memory is consolidated. Memory con-solidation in the brain involves the process of encoding informationin a long-term manner via strengthening of synaptic pathways andbrain regions that activate in response to that information (1, 71).These long-term modifications can last from hours up to an entirelifetime (1, 60).In NWNs, PRL allows for strengthening specific pathways overtime (and weakening of others), based on a desired output. Oncepathways are consistently and repeatedly activated, they takenotably longer to decay. In experiment, 3-hour rest was allocatedbetween trials for the physical network. However, this was likelynot long enough for conductance pathways to fully decay, particu-larly once PRL was introduced. NWNs have previously been shownto retain information even 24 hours after dynamic pathway tuning(38). Consequently, after reinforcement or repeated prolonged acti-vation of specific pathways, NWNs’ memory for those pathways isalso lengthened and consolidated. In contrast, memristive junctionsand pathways in simulated NWNs were completely reset betweeneach trial and displayed a lowerWM capacity. These results are con-sistent with findings by Benna and Fusi (39), which suggest that syn-aptic plasticity depends on the history of synaptic modifications,referred to as synaptic metaplasticity. In physical NWNs, memoryof previous junction modifications is carried on between epochsand trials more effectively than in simulation, increasing the WMcapacity of the network.Similar behavior was previously reported in Ag-PVP NWNs byMilano et al. (72), who found that the structural topology of NWNsevolves depending on synaptic history. In that study, however, re-routing of conductance pathways was demonstrated by applyingsufficiently high current densities to rupture physical connectionsbetween wires. In contrast, the present study uses much lower volt-ages, which maintains persistent activity in the network. This isidentified with WM (1). Synaptic metaplasticity as described hereis a result of external feedback signals into the network ratherthan physical restructuring.In task 3, the network was charged with only retaining pathwayinformation for one target pattern. While the NWNs still had tocontend with six interference patterns and therefore provided acomprehensive insight into the WM capabilities of the networks,the capacity for multiple classes to be held in memory and recalledwas not measured. Consequently, NWNs demonstrate stimulus-specific manipulation, while WM in humans also involvesdomain-specific manipulation (73). The latter of these wouldrequire memory across multiple classes of stimuli, not just asingle target pattern. To properly mimic large-scale parallel infor-mation manipulation in the brain, future studies into the network’scapacity to remember and recall multiple pathways associated withdifferent input patterns are warranted. However, it may be that mul-tiple, highly modular NWNs will be required to be linked up in par-allel to demonstrate such information processing abilities (21).While NWNs are highly scalable as they are straightforwardly syn-thesized by bottom-up self-assembly, device scalability is limited byfabrication of the multielectrode system. Previously, other NWNdevices have been fabricated in CMOS multielectrode arrays(MEAs) for implementing reservoir computing. These deviceshave not shown marked performance improvements when scalingfrom a 16-electrode MEA (10, 18, 23) to a 64-electrode MEA (74).The present study implements cognitive tasks rather than reservoircomputing, and therefore an increased number of electrodes wouldallow demonstration of the n-back WM task with more complexpatterns.Neuromorphic systems that can learn, remember, and adapt toexternal time-varying stimuli would represent a breakthrough plat-form for neuro-inspired computing (75). The present study demon-strates the potential for NWNs to achieve this. The ability to processdynamically changing information is key in many real world appli-cations, such as robotics and sensor edge devices, where there is aneed to make on-the-fly decisions in a nondeterministic environ-ment (76) .In conclusion, by applying supervised and reinforcement learn-ing strategies similar to those operating in the brain, we have dem-onstratedWM and memory consolidation in NWNs. These higher-order cognitive functions were achieved by implementing a non-trivial cognitive task routinely applied to human subjects. Resultsreveal that neuromorphic learning paradigms implemented inNWNs leverage similar mechanisms to the brain, namely, synapticmetaplasticity and synaptic strengthening and pruning, to optimizeWM and memory consolidation.METHODSExperimental setupSilver NWNs were synthesized by following the well-known Polyolmethod, as described previously (14), which produces an ethanolsolution of Ag nanowires coated with the polymer PVP. Nanowireswere directly deposited by drop-casting on a glass substrate to createdense and homogeneous networks of interconnected Ag-PVPnanowires, resulting from the random dispersion of nanowiresonce the ethanol droplet evaporates. A multielectrode Ag-PVPNWN device (Fig. 1A) was constructed by depositing two regulararrays of rectangular gold electrodes facing each other at a distanceof 3 mm. Electrodes were 200 μm wide and 600 μm apart from eachother. The electrodes were deposited onto the glass substrate bymagnetron sputtering before depositing the nanowires. Dependingon the task, a selection of electrodes out of the two arrays were con-nected to serve, respectively, as source or drain electrodes. The elec-tronic setup is similar to the one described in Diaz-Alvarez et al.(41), comprising a digitally controlled switch box connected tothe electrodes in use for the given task, which sequentially opensand closes the respective electrodes that form the patterns used inLoeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 8 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023all the tasks (cf. Fig. 1B). Source voltage was delivered and con-trolled with a National Instruments data aqcuisition (NI-DAQ)card, and drain current was controlled using an array of in-housebiased operational amplifiers (OPAs), serving as current-voltageconverters with amplification of 107 V/A. Voltage signal from theOPA array was sent to a digital acquisition card (NI-DAQ). In alltasks and experiments, the rate of acquisition was the same (1 kHz).This rate was further down-sampled for storage, analysis, and pre-sentation purposes. Software to control voltage sources, digitalswitches, the acquisition cards, and the different algorithms fortraining/testing was developed using Python.Full characterization of Ag-PVP NWNs, including the mecha-nisms of activation and decay are reported in our previous studies(9, 14, 38, 41), where we found that memory of previous conduc-tance pathways begins to fade at 35 to 90 s, although networkscan take much longer to decay (up to hours) due to many junctionsin different states. Using the same multiterminal (nine input andnine output electrode) device configuration, Diaz-Alvarez et al.(41) demonstrated the ability to dynamically control conductancepathways in Ag-PVP NWNs by maintaining low input voltagelevels, effectively keeping junctions in intermediate “metastable”states between low and high conductance. This was similarly laterdemonstrated by Li et al. (38) using the same electrode configura-tion but with Ag@TiO2 NWNs. Consequently, the present studyadopts a similar protocol, using low voltages during training andtesting (0.1 to 0.2 V), and input samples are presented for relativelyshort durations (within the ∼1-min memory window). Networkswere also rested for 3 hours between trials to allow decay of conduc-tance pathways. Further characterization is shown in figure 3 andfigure S3 of the study of Diaz-Alvarez et al. (14), although theyused double-probe electrodes rather than a multiterminal device.Two unique network samples were prepared and assessed for thedifferent tasks, each with different input patterns and order ofsample presentation. While results do vary from device to device(see source data file S1), they tend to follow similar trends withina range [see also Diaz-Alvarez et al. (14)] (fig. S1).Simulation setupAg-PVP NWNs were modeled as described in previous studies (9,12, 21). All simulations were conducted using Python. Briefly,nanowires were modeled as one-dimensional (1D) objects oflength drawn from a gamma distribution (mean wire length =10 μm), placed randomly within a 2D plane of fixed size (75 μmby 75 μm). The vertical and horizontal positions of wires were gen-erated from a uniform distribution on 2π. In this study, NWN sim-ulations used 698 nanowires (nodes) and 2582 junctions (edges),giving an average degree of 7.40. While the number of modelednanowires is smaller than in the experimental network, NWNmemory capacity is determined by the number of junctions,which was chosen to achieve simulation results that most closelymatched experimental measurements. Previous NWN simulationstudies (12, 15, 21) using the same model demonstrate the influenceof topology and density on network functionality.Nanowire-nanowire cross-points were modeled as ideal, electri-cally insulating, ionically conducting junctions, with threshold-driven bipolar memristive switching (14, 17, 59, 77, 78), modulatedby electron tunnelling transport (9, 15). Junction conductance, Gj =Gj(λ), depends on a state variable λ(t) that parametrizes the con-ducting nanofilament responsible for memristive switching due toelectrochemical metallization (6). The evolution of λ(t) is describedby a polarity-dependent voltage threshold model (9, 15, 59, 78, 79)as shown in Eq. 1dλdt¼ðjVðtÞj � VsetÞ sgn½VðtÞ�; jVðtÞj. Vset0; Vreset ,jVðtÞj, VsetbðjVðtÞj � VresetÞ sgn½λðtÞ�; jVðtÞj, Vreset8<:ð1Þwhere V is the voltage across a junction, and Vset and Vreset are thejunction on and off thresholds, respectively. A positive constant pa-rameter b was also used to quantify the filament decay rate due tostochastic thermodynamic breakdown (80–82). This parameter wasvaried (see fig. S3) to explore its effect on WM. For simulationresults shown in this paper, b = 0.5 was used, unless otherwiseindicated.Experimental validation of the model used to simulate these net-works can be found in Hochstetter et al. (9) (figs. S19 and S20). Thechoice of model parameters used in this study is based on an exten-sive parameter sensitivity analysis by Hochstetter and colleagues (9)in their Supplementary Notes.The simulation duration and time steps used in all tasks weret = 2 s (for each sample) and Δt = 0.01 s, respectively. Inputvoltage amplitude was 0.3 V for training and 0.1 V for testing.For simulation results, the same network was used 10 times, eachwith different input patterns and order of sample presentation.Network junction states were completely reset between trials insimulation.Implementation of supervised learning and PRL inclassification and WM tasksFigure 6 summarizes the three variations of n-back task presented inthis study. For tasks 1 and 2, the target pattern is randomly switchedbetween A and B in each epoch, and for task 3, the position of thetarget A is semirandomly changed at each training-testing epoch.Algorithm 1 describes the methodology for one training-testingepoch (for tasks 1 and 2), while Table 1 lists and describes all theparameters. A gradient descent–like algorithm (see Eq. 2) wasused during training to implement supervised learning andchange the relative voltage between the input and output electrodes,by increasing or decreasing the output voltage (Vo) of the drain elec-trodes. Changes in Vo are described by the following equation pairon lines 17 and 18 of Algorithm 1, respectivelyΔ ¼ βðytarget � dtargetÞ ð2ÞVo ¼ Vo þ Δ ð3Þwhere ytarget is the target drain output (current), dtarget is the settarget current, and β is the learning rate. Both ytarget and dtargetare normalized and therefore dimensionless, meaning that Δ isalso dimensionless. By changing Vo, corresponding output draincurrents were nudged toward a target current threshold (θ). Inputelectrodes remained unchanged from Vi. Experimentally, trainingstops once ytarget reaches θ or the training time per sample ends.In the latter case, the sample was stopped even if the current hadnot reached the threshold θ, and the next sample was presented.The maximum training time was set to t = 15 s. If θ was reachedbefore 15 s (which typically occurred after 1 to 3 s), the samplewas considered trained, training was halted, and the next sampleLoeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 9 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023was presented. In simulation, each training samplewas shown for allt = 2 s (200 time steps). If θ was reached before 2 s,Vi and allVo werereset to zero for the rest of the sample duration.During testing, all the output electrodes were opened and reset to0 V. An input that matches the same pattern as sample 1 (i.e., nsamples before testing) was then delivered to the network. Theinput voltage was ramped up from 0 V until a current threshold(θtest) is reached at the drain electrodes or until the testing time (typ-ically t = 7 s) ends. The voltage does not exceed xtest. This procedureis used only in the experimental setup, as it is difficult to know apriori how much voltage is required for drain current to be measur-able. In simulation, however, drain currents can be measured forany arbitrary voltage, so samples were delivered at a set voltage(Vi = xtest), which is lower than training.Four possible scenarios can occur as a result of supervisedlearning1) If ytarget < dtarget, then Δ < 0. Therefore, based on Eq. 3, Vo isreduced for the next time step. This, in turn, means that the voltagedifference between the input electrodes (Vi) and Votarget decreases,and more current flows to the target drain than the other drainelectrodes.2) If ytarget ≥ dtarget, then θ is reached, and the present sample istrained. Voltages are reset to 0.3) If ynontarget < dtarget, the current outputs from the nontargetdrains are likely lower than the outputs from the target drains.4) If ynontarget ≥ dtarget, in this scenario, Δ ≥ 0, meaning thatVonontarget increases, and less current flows to the nontarget drains.Once the test sample ended, the current of each output channelduring testing timewas averaged. The channel for which the averagecurrent was greater is considered the winner of the epoch (i.e.,argmax is applied). If the winning output electrode matches thetarget that was trained for the testing electrode pattern (i.e., the cor-responding drain from sample 1), then it is considered a successfulepoch (Acc = 1), and the current threshold level (θ) stays the same.Otherwise, the epoch is considered unsuccessful (Acc = 0), and θ isincreased for the target drain (reinforcement), while the thresholdof the nontarget drain is decreased (penalty).Task 1A simple binary classification task was implemented (using fourinputs and two outputs) to demonstrate supervised learning andPRL in NWNs. Each of two nonoverlapping 2 by 2 sample grid pat-terns (A and B) was associated with a corresponding groundedoutput electrode (drain 1 or 2) with a one-to-one correspondence.Each pattern was input as a voltage bias (Vi) applied to the selectedsource electrodes for t = 2 s, while the other input electrodes wereelectromechanically closed. The target output electrode wasopened, while the other output electrode was electromechanicallyclosed. After n = 2 cycles of training, a testing sequence was per-formed, in which the network’s efficacy in reproducing thetrained pattern that presented n-steps previously was measuredand analyzed. Training and testing for the selected input-outputpatterns were performed in sequence over multiple training-testing epochs.In experiment, parameters used for task 1 were as follows: xtrain =0.2 V; for xtest, voltage was ramped up from 0 V until a current wasFig. 6. Summary of n-back task protocols. Top (task 1): Binary classification oftwo 2 × 2 patterns (A and B; two nonzero inputs each) and n = 2. Middle (task2): Binary classification of 3 × 3 patterns (five nonzero inputs) and varying n.Bottom (task 3): WM task using multiple randomly selected, nonrepeated 3 × 3patterns (A to G; example sequence shown) and varying n. Characters in red rep-resent target pattern tested in each task.Table 1. Experimental and simulation parameters and variables.Parameter Descriptiony Normalized current (I ) outputs of the drains, vector [ytarget,ynontarget]d Normalized current (I ) that target drain is trained toreach (ytarget)Vo Output (drain) voltages (1 × 2 vector)Io Nonnormalized output (drain) currents (1 × 2 vector)Vi Input voltage (scalar)β Learning rateλ A state variable that parametrizes the conducting filamentresponsible for memristive switching∆ Amount of adjustment required to Vo at each time stepb Junction filament decay parameter (lower = slower decay)Acc Classification accuracyAccθ Accuracy threshold for reinforcementθ Reinforcement learning threshold, vector [θtarget,θnontarget].Once the target output current (ytarget) reaches thisthreshold, it is considered trained for that sample.θtest Current threshold for testing used only in experimentalsetup. Once the current reaches this threshold, testingis halted.incV al Increase θtarget by incV al when Acc < AccθdecV al Decrease θnontarget by decV al when Acc < Accθxtrain Input voltage during trainingxtest Input voltage during testing, with xtest ≪ xtrain, so that newpathways are not formed during testing.Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 10 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023measurable in the target drain; θ= 6 × 10−8 A; θtest= 1 × 10−7 A;training time: t = 15 s or until θ was reached; testing time: t = 7 sor until θtest was reached, with 7-s rest between each training sampleand 1-s rest between training and testing. In simulation, parametersused were as follows: xtrain = 0.3 V; xtest = 0.1 V; θ= 5 × 10−6 A; train-ing time: t = 2 s; testing time: t = 2 s, with no rest between eachtraining sample and no rest between training and testing.Task 2The input pattern was expanded to 3 × 3 (nine inputs, two outputs),and more n values were added to the testing procedure: n = 2,3,4,5,and 6. Training and testing were implemented in the same way as inAlgorithm 1, using more complex input patterns: “x” and “+” (5 bitseach). One training-testing cycle made up an epoch. Each triallasted 40 epochs of one n value. For example, in trial 1, n = 2, intrial 2, n = 3, and so on. A total of 50 trials were run, 10 for eachn value, each with a different random seed. Figure 6 shows a sche-matic of the setup and methodology. The two classes of 3 × 3 pat-terns (+ and x) correspond to five nonzero inputs. If sample 1 is +,subsequent training samples were all x and vice versa. This way,during testing, conduction pathways formed by sample 1 are notstimulated for n-1 samples beforehand. Thus, the network relieson memory of pathways activated by the target input.In experiment, parameters used for task 2 were as follows: xtrain =0.1 V; for xtest, voltage was ramped up from 0 V until a current wasmeasurable in the target drain; θ= 2 × 10−8 A; θtest= 1 × 10−7 A;training time: t = 15 s or until θ was reached; testing time: t = 6 sor until θtest was reached, with 5-s rest between each training sampleand 5-s rest between training and testing. In simulation, parametersused were as follows: xtrain = 0.3 V; xtest = 0.1 V; θ= 5 × 10−6 A; train-ing time: t = 2 s; testing time: t = 2 s, with no rest between eachtraining sample and no rest between training and testing.Task 3A multiple pattern n-back task was implemented to test the WMcapacity of NWNs while minimizing the influence of previoustrials. This was operationalized by testing how accurately theNWN recalls a test sample from n-steps ago, without beingAlgorithm 1: Binary classification procedure1: n ⇐ 2; d ⇐ 12: xtrain ⇐ 0.3 V; xtest ⇐ 0.1 V3: incV al ⇐ θ/3; decV al ⇐ θ/64: θ⇐ [0.5, 0.5]5: Vo ⇐ [0, 0]6: for s in range(length(samples)) do7: if TRAINING then8: target ⇐ random (1, 2)▷ Choose a class (1 or 2) at random and set astarget9: Vi ⇐ xtrain10: Close Vonontarget11: RUN SIMULATION12: y⇐ Normalize(Io)13: if ytarget > θtarget then ▷ If θ is reached, reset voltages to 014: Vo ⇐ [0, 0]15: Vi ⇐ 016: else17: ∆ ⇐ β(ytarget − dtarget) ▷ Supervised learning18: Vo ⇐ Vo + ∆▷ External feedback19: end if20: else if TESTING then21: Vi⇐ xtest22: Vo ⇐ [0, 0]23: target ⇐ samples[s − n] ▷ Target pattern is same as pattern ntraining samples prior.24: Open Vonontarget25: RUN SIMULATION26: y ⇐ Normalize(Io)27: if mean(ytarget) > mean(ynontarget) then28: Acc ⇐ 129: else30: Acc ⇐ 031: end if32: if Acc < Accθ then ▷ Reinforcement (PRL)33: θtarget ⇐ θtarget + incV al34: θnontarget ⇐ θnontarget − decV al35: end if36: end if37: end forAlgorithm 2: Multipattern n-back procedure1: patterns ⇐ [A, B, C, D, E, F, G]2: d ⇐ 13: xtrain ⇐ 0.3 V; xtest ⇐ 0.1 V4: incV al ⇐ θ/6; decV al ⇐ θ/125: θ ⇐ [0.5, 0.5]6: Vo ⇐ [0, 0]7: nV als ⇐ sample(length(patterns), replace = False) ▷ Randomly samplepositions at which target pattern (A) will be located. Repeat around 28times for a total of 200 positions.8: for s in range(length(samples)) do9: if TRAINING then10: target ⇐ A11: for pattern in patterns do12: n⇐ nV alss13: Vi ⇐ xtrain14: Close Vonontargets15: RUN SIMULATION16: y⇐ Normalize(Io)17: if ytarget > θtarget then ▷ If θ is reached, reset voltages to 018: Vo ⇐ [0, 0]19: Vi ⇐ 020: else21: ∆ ⇐ β(ytarget − dtarget) ▷ Supervised learning22: Vo ⇐ Vo + ∆▷ External feedback23: end if24: end for25: else if TESTING then26: Vi ⇐ xtest27: Vo ⇐ [0, 0]28: target ⇐ A29: Open Vonontargets30: RUN SIMULATION31: y ⇐ Normalize(Io)32: if mean(ytarget) > mean(ynontargets) then33: Acc ⇐ 134: else35: Acc ⇐ 036: end if37: if Acc < Accθ then ▷ Reinforcement (PRL)38: θtarget ⇐ θtarget + incV al39: θnontargets ⇐ θnontargets − decV al40: end if41: end if42: end forLoeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 11 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023reminded of that sample within the same epoch. To overcome thepotential extra training of pathways during a single epoch due torepeating patterns (e.g., “A” repeated four times in the n-back = 5pattern “ABBBBA” shown in Fig. 6), random, distinct patternswere generated.Here, seven random 3 × 3 patterns were generated (named pat-terns A, B, C,…G). The number of pixels per pattern was limited tothree, so that each pattern could have 1, 2, or 3 pixels randomly se-lected as inputs from the nine available pixels. The supervised learn-ing and PRL algorithm used for one epoch (training + testing) ispresented in Algorithm 2.Pattern A was selected as the target for each testing epoch in theexperiment. To vary n, the position at which pattern Awas present-ed during training was varied, so that for testing, n-steps back varies.For example, if n = 3, pattern A is presented as the fifth trainingsample. In this case, the other six samples (four presented beforeA and two after) are randomly selected from patterns B to Gwithout replacement. This meant that each training epoch includedadditional nonrepeated samples that precede the n-back sequence.In the example sequence shown in Fig. 6, the training order is [B, E,F, G, A, C, and D], which is followed by testing only for [A].One training-testing cycle made up an epoch. Each trial lasted200 epochs. The location of pattern A was semirandomized, sothat each n was sampled around 28 times, for a total of 200epochs. In other words, pattern Awas presented at training position1, 28 times over 200 epochs, at position 2, 28 times, and so on. Atotal of 10 trials were run, each with a different random seed.In experiment, parameters used for task 3 were as follows: xtrain =0.1 V; for xtest, voltage was ramped up from 0 V until a current wasmeasurable in the target drain; θ= 6 × 10−8 A; θtest= 1 × 10−7 A;training time: t = 15 s or until θ was reached; testing time: t = 6 sor until θtest was reached, with 5-s rest between each training sampleand 5-s rest between training and testing. In simulation, parametersused were as follows: xtrain = 0.3 V; xtest = 0.1 V; θ= 5 × 10−6 A; train-ing time: t = 2 s; testing time: t = 2 s, with no rest between eachtraining sample and no rest between training and testing.Supplementary MaterialsThis PDF file includes:Supplementary TextFigs. S1 to S7Legends for movies S1 to S4Legend for Source Data File S1Other Supplementary Material for thismanuscript includes the following:Movies S1 to S4Source Data File S1View/request a protocol for this paper from Bio-protocol.REFERENCES AND NOTES1. P. Dayan, L. F. Abbott, Theoretical Neuroscience: Computational and Mathematical Modelingof Neural Systems (MIT Press, 2005).2. L. Luo, Architectures of neuronal circuits. Science 373, eabg7285 (2021).3. H. Markram, E. Muller, S. Ramaswamy, M. W. Reimann, M. Abdellah, C. A. Sanchez,A. Ailamaki, L. Alonso-Nanclares, N. Antille, S. Arsever, G. A. A. Kahou, T. K. Berger, A. Bilgili,N. Buncic, A. Chalimourda, G. Chindemi, J. D. Courcol, F. Delalondre, V. Delattre,S. Druckmann, R. Dumusc, J. Dynes, S. Eilemann, E. Gal, M. E. Gevaert, J. P. Ghobril, A. Gidon,J. W. Graham, A. Gupta, V. Haenel, E. Hay, T. Heinis, J. B. Hernando, M. Hines, L. Kanari,D. Keller, J. Kenyon, G. Khazen, Y. Kim, J. G. King, Z. Kisvarday, P. Kumbhar, S. Lasserre, J. V. leBé, B. R. C. Magalhães, A. Merchán-Pérez, J. Meystre, B. R. Morrice, J. Muller, A. Muñoz-Céspedes, S. Muralidhar, K. Muthurasa, D. Nachbaur, T. H. Newton, M. Nolte, A. Ovcharenko,J. Palacios, L. Pastor, R. Perin, R. Ranjan, I. Riachi, J. R. Rodríguez, J. L. Riquelme, C. Rössert,K. Sfyrakis, Y. Shi, J. C. Shillcock, G. Silberberg, R. Silva, F. Tauheed, M. Telefont, M. Toledo-Rodriguez, T. Tränkler, W. van Geit, J. V. Díaz, R. Walker, Y. Wang, S. M. Zaninetta, J. DeFelipe,S. L. Hill, I. Segev, F. Schürmann, Reconstruction and simulation of neocortical microcir-cuitry. Cell 163, 456–492 (2015).4. N. Brunel, Dynamics of sparsely connected networks of excitatory and inhibitory spikingneurons. J. Comput. Neurosci. 8, 183–208 (2000).5. L. E. Suárez, B. A. Richards, G. Lajoie, B. Misic, Learning function from structure in neuro-morphic networks. bioRxiv , 2020.11.10.350876 (2020).6. Z. Kuncic, T. Nakayama, Neuromorphic nanowire networks: Principles, progress and futureprospects for neuro-inspired information processing. Adv. Phys. X 6, 1894234 (2021).7. W. Zhang, Neuro-inspired computing chips. Nat. Electron. 3, 371–382 (2020).8. A. Mehonic, A. Sebastian, B. Rajendran, O. Simeone, E. Vasilaki, A. J. Kenyon, Memristors—From in-memory computing, deep learning acceleration, and spiking neural networks tothe future of neuromorphic and bio-inspired computing. Adv. Intell. Syst. 2,2000085 (2020).9. J. Hochstetter, R. Zhu, A. Loeffler, A. Diaz-Alvarez, T. Nakayama, Z. Kuncic Avalanches andedge-of-chaos learning in neuromorphic nanowire networks. Nat. Commun. 12(1),4008 (2021).10. C. S. Dunham et al., Nanoscale neuromorphic networks and criticality: A perspective.J. Phys. Complex. 2, 042001 (2021).11. G. Milano, S. Porro, I. Valov, C. Ricciardi, Recent developments and perspectives formemristive devices based on metal oxide nanowires. Adv. Electron. Mater. 5,1800909 (2019).12. A. Loeffler, R. Zhu, J. Hochstetter, M. Li, K. Fu, A. Diaz-Alvarez, T. Nakayama, J. M. Shine,Z. Kuncic, Topological properties of neuromorphic nanowire networks. Front. Neurosci. 14,184 (2020).13. A. Z. Stieg, A. V. Avizienis, H. O. Sillin, R. Aguilera, H. H. Shieh, C. Martin-Olmos, E. J. Sandouk,M. Aono, J. K. Gimzewski, in Self-organization and Emergence of Dynamical Structures inNeuromorphic Atomic Switch Networks, A. Adamatzky, L. Chua, Eds. (Springer, Cham, 2014),Memristor Networks, pp. 391–427.14. A. Diaz-Alvarez, R. Higuchi, P. Sanz-Leon, I. Marcus, Y. Shingaya, A. Z. Stieg, J. K. Gimzewski,Z. Kuncic, T. Nakayama, Emergent dynamics of neuromorphic nanowire networks. Sci. Rep.9, 14920 (2019).15. R. Zhu, J. Hochstetter, A. Loeffler, A. Diaz-Alvarez, T. Nakayama, J. T. Lizier, Z. Kuncic, In-formation dynamics in neuromorphic nanowire networks. Sci. Rep. 11, 13047 (2021).16. A. V. Avizienis, H. O. Sillin, C. Martin-Olmos, H. H. Shieh, M. Aono, A. Z. Stieg, J. K. Gimzewski,Neuromorphic atomic switch networks. PLOS ONE 7, e42772 (2012).17. H. O. Sillin, R. Aguilera, H. H. Shieh, A. V. Avizienis, M. Aono, A. Z. Stieg, J. K. Gimzewski, Atheoretical and experimental study of neuromorphic atomic switch networks for reservoircomputing. Nanotechnology 24, 384004–384015 (2013).18. E. C. Demis, R. Aguilera, H. O. Sillin, K. Scharnhorst, E. J. Sandouk, M. Aono, A. Z. Stieg,J. K. Gimzewski, Atomic switch networks - Nanoarchitectonic design of a complex systemfor natural computing. Nanotechnology 26, 204003 (2015).19. K. Fu, R. Zhu, A. Loeffler, J. Hochstetter, A. Diaz-Alvarez, A. Stieg, J. Gimzewski, T. Nakayama,Z. Kuncic, Reservoir computing with neuromemristive nanowire networks, in Proceedingsof the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 19 to 24July 2020.20. R. Zhu, J. Hochstetter, A. Loeffler, A. Diaz-Alvarez, A. Stieg, J. Gimzewski, T. Nakayama,Z. Kuncic, Harnessing adaptive dynamics in neuro-memristive nanowire networks fortransfer learning, in Proceedings of the 2020 International Conference on Rebooting Com-puting (ICRC), Atlanta, GA, USA, 1 to 3 December 2020.21. A. Loeffler, R. Zhu, J. Hochstetter, A. Diaz-Alvarez, T. Nakayama, J. M. Shine, Z. Kuncic,Modularity and multitasking in neuro-memristive reservoir networks. NeuromorphicComput. Eng. 1, 014003 (2021).22. R. Zhu, A. Loeffler, J. Hochstetter, A. Diaz-Alvarez, T. Nakayama, A. Stieg, J. Gimzewski,J. Lizier, Z. Kuncic,MNIST Classification Using Neuromorphic Nanowire Networks (Associationfor Computing Machinery, New York, NY, USA, 2021).23. S. Lilak, W. Woods, K. Scharnhorst, C. Dunham, C. Teuscher, A. Z. Stieg, J. K. Gimzewski,Spoken digit cclassification by in-materio reservoir computing with neuromorphic atomicswitch networks. Tech. Rep. (2021).24. W. Maass, T. Natschläger, H. Markram, Real-time computing without stable states: A newframework for neural computation based on perturbations. Neural Comput. 14,2531–2560 (2002).25. H. Jaeger, Short term memory in echo state networks. GMD-Report 152. GMD-GERMANNATIONAL RESEARCH INSTITUTE FOR COMPUTER SCIENCE (2002).Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 12 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023https://en.bio-protocol.org/cjrap.aspx?eid=10.1126/sciadv.adg328926. D. Verstraeten, B. Schrauwen, M. D’Haene, D. Stroobandt, An experimental unification ofreservoir computing methods. Neural Netw. 20, 391–403 (2007).27. G. Tanaka, T. Yamane, J. B. Héroux, R. Nakane, N. Kanazawa, S. Takeda, H. Numata,D. Nakano, A. Hirose, Recent advances in physical reservoir computing: A review. NeuralNetw. 115, 100–123 (2019).28. K. Doya, What are the computations of the cerebellum, the basal ganglia and the cerebralcortex? Neural Netw. 12, 961–974 (1999).29. E. I. Knudsen, Supervised learning in the brain. J. Neurosci. 14, 3985–3997 (1994).30. O. Baumann, R. J. Borra, J. M. Bower, K. E. Cullen, C. Habas, R. B. Ivry, M. Leggio,J. B. Mattingley, M. Molinari, E. A. Moulton, M. G. Paulin, M. A. Pavlova, J. D. Schmahmann,A. A. Sokolov, Consensus paper: The role of the cerebellum in perceptual processes. Cer-ebellum 14, 197–220 (2015).31. A. A. Sokolov, R. C. Miall, R. B. Ivry, The cerebellum: Adaptive prediction for movement andcognition. Trends Cogn. Sci. 21, 313–332 (2017).32. J. D. Schmahmann, X. Guell, C. J. Stoodley, M. A. Halko, The theory and neuroscience ofcerebellar cognition. Annu. Rev. Neurosci. 42, 337–364 (2019).33. J. C. Houk, J. L. Davis, D. G. Beiser, in A Model of How the Basal Ganglia Generate and UseNeural Signals That Predict Reinforcement (MIT Press, 1994), pp. 249–270.34. M. G. Packard, B. J. Knowlton, Learning and memory functions of the basal ganglia. Annu.Rev. Neurosci. 25, 563–593 (2002).35. W. Schultz, L. Tremblay, J. R. Hollerman, Reward processing in primate orbitofrontal cortexand basal ganglia. Cereb. Cortex 10, 272–284 (2000).36. T. V. Maia, Reinforcement learning, conditioning, and the brain: Successes and challenges.Cogn. Affect. Behav. Neurosci. 9, 343–364 (2009).37. T. V. Maia, M. J. Frank, From reinforcement learning models to psychiatric and neurologicaldisorders. Nat. Neurosci. 14, 154–162 (2011).38. Q. Li, A. Diaz-Alvarez, R. Iguchi, J. Hochstetter, A. Loeffler, R. Zhu, Y. Shingaya, Z. Kuncic,K. I. Uchida, T. Nakayama, Dynamic electrical pathway tuning in neuromorphic nanowirenetworks. Adv. Funct. Mater. 30, 2003679 (2020).39. M. K. Benna, S. Fusi, Computational principles of synaptic memory consolidation. Nat.Neurosci. 19, 1697–1706 (2016).40. J. L. Raymond, J. F. Medina, Computational principles of supervised learning in the cere-bellum. Annu. Rev. Neurosci. 41, 233–253 (2018).41. A. Diaz-Alvarez, R. Higuchi, Q. Li, Y. Shingaya, T. Nakayama, Associative routing throughneuromorphic nanowire networks. AIP Adv. 10, 025134 (2020).42. M. J. Kane, R. W. Engle, The role of prefrontal cortex in working-memory capacity, executiveattention, and general fluid intelligence: An individual-differences perspective. Psychon.Bull. Rev. 9, 637–671 (2002).43. M. J. Kane, A. R. A. Conway, T. K. Miura, G. J. H. Colflesh, Workingmemory, attention control,and the N-back task: A question of construct validity. J. Exp. Psychol. Learn. Mem. Cogn.33(3), 615–622 (2007).44. S. M. Jaeggi, M. Buschkuehl, W. J. Perrig, B. Meier, The concurrent validity of the N-back taskas a working memory measure. Memory 18, 394–412 (2010).45. C. C. Hilgetag, A. Goulas, Is the brain really a small-world network? Brain Struct. Funct. 221,2361–2366 (2016).46. J. Jonides, E. H. Schumacher, E. E. Smith, E. J. Lauber, E. Awh, S. Minoshima, R. A. Koeppe,Verbal working memory load affects regional brain activation as measured by PET. J. Cogn.Neurosci. 9(4), 462–475 (1997).47. A. Baddeley, Working memory. Science 255, 556–559 (1992).48. N. Cowan, Chapter 20: What are the differences between long-term, short-term, andworking memory? Prog. Brain Res. 169, 323–338 (2008).49. D. J. Amit, S. Fusi, Learning in neural networks with material synapses. Neural Comput. 6,957–982 (1994).50. S. Fusi, L. F. Abbott, Limits on the memory storage capacity of bounded synapses. Nat.Neurosci. 10, 485–493 (2007).51. A. Pouget, P. Dayan, R. Zemel, Information processing with population codes. Nat. Rev.Neurosci. 1, 125–132 (2000).52. A. M. Zador, A critique of pure learning and what artificial neural networks can learn fromanimal brains. Nat. Commun. 10, 3770 (2019).53. E. Neftci, J. Binas, U. Rutishauser, E. Chicca, G. Indiveri, R. J. Douglas, Synthesizing cognitionin neuromorphic electronic systems. Proc. Natl. Acad. Sci. U.S.A. 110, E3468–E3476 (2013).54. A. L. Yuille, N. M. Grzywacz, A winner-take-all mechanism based on presynaptic inhibitionfeedback. Neural Comput. 1, 334–347 (1989).55. H. G. Manning, F. Niosi, C. G. da Rocha, A. T. Bellew, C. O’Callaghan, S. Biswas, P. F. Flowers,B. J. Wiley, J. D. Holmes, M. S. Ferreira, J. J. Boland, Emergence of winner-takes-all con-nectivity paths in random nanowire networks. Nat. Commun. 9, 3219 (2018).56. C. O’Callaghan, C. G. Rocha, F. Niosi, H. G. Manning, J. J. Boland, M. S. Ferreira, Collectivecapacitive and memristive responses in random nanowire networks: Emergence of criticalconnectivity pathways. J. Appl. Phys. 124, 152118 (2018).57. B. Scellier, S. Mishra, Y. Bengio, Y. Ollivier, Agnostic physics-driven deep learning (2022).58. M. N. Boon, H.-C. Ruiz Euler, T. Chen, B. van de Ven, U. A. Ibarra, P. A. Bobbert, W. G. van derWiel, Gradient descent in materio. arXiv:2105.11233 [cs.NE] (15 May 2021).59. Z. Kuncic, I. Marcus, P. Sanz-Leon, R. Higuchi, Y. Shingaya, M. Li, A. Stieg, J. Gimzewski,M. Aono, T. Nakayama, Emergent brain-like complexity from nanowire atomic switchnetworks: Towards neuromorphic synthetic intelligence, in Proceedings of the 2018 IEEE18th International Conference on Nanotechnology (IEEE-NANO), Cork, Ireland, 23 to 26 2018July 2018.60. N. Deperrois, M. Graupner, Short-term depression and long-term plasticity together tunesensitive range of synaptic plasticity. PLOS Comput. Biol. 16, e1008265 (2020).61. W. K. Kirchner, Age differences in short-term retention of rapidly changing information.J. Exp. Psychol. 55, 352–358 (1958).62. A. Goulas, F. Damicelli, C. C. Hilgetag, Bio-instantiated recurrent neural networks: Inte-grating neurobiology-based network topology in artificial networks. Neural Netw. 142,608–618 (2021).63. G. A. Miller, The magical number seven, plus or minus two: Some limits on our capacity forprocessing information. Psychol. Rev. 63, 81–97 (1956).64. N. Cowan, The magical number 4 in short-term memory: A reconsideration of mentalstorage capacity. Behav. Brain Sci. 24(1), 87–114 (2001).65. N. Cowan, The magical mystery four: How is working memory capacity limited, and why?Curr. Dir. Psychol. Sci. 19, 51–57 (2010).66. W. J. Chai, A. I. Abd Hamid, J. M. Abdullah, Working memory from the psychological andneurosciences perspectives: A review. Front. Psychol. 9, 401 (2018).67. G. Mongillo, O. Barak, M. Tsodyks, Synaptic theory of working memory. Science 319,1543–1546 (2008).68. W. Schultz, Reward functions of the basal ganglia. J. Neural Transm. 123, 679–693 (2016).69. J. C. Tapia, J. W. Lichtman, Synapse elimination, in Fundamental Neuroscience, L. Squire,D. Berg, F. E. Bloom, S. du Lac, A. Ghosh, N. C. Spitzer, Ed. (Elsevier Science, ed. 4, 2013),pp. 437–455.70. G. Chechik, I. Meilijson, E. Ruppin, Neuronal regulation: A mechanism for synaptic pruningduring brain maturation. Neural Comput. 11, 2061–2080 (1999).71. D. Hebb, The organization of behavior: A neuropsychological theory, in The Organization ofBehavior (2005); https://doi.org/10.4324/9781410612403.72. G. Milano, G. Pedretti, M. Fretto, L. Boarino, F. Benfenati, D. Ielmini, I. Valov, C. Ricciardi,Brain-inspired structural plasticity through reweighting and rewiring in multi-terminal self-organizing memristive nanowire networks. Adv. Intell. Syst. 2, 2000096 (2020).73. D. Li, S. E. Christ, N. Cowan, Domain-general and domain-specific functional networks inworking memory. Neuroimage 102, 646–656 (2014).74. E. C. Demis, R. Aguilera, K. Scharnhorst, M. Aono, A. Z. Stieg, J. K. Gimzewski, Nanoarchi-tectonic atomic switch networks for unconventional computing. Jpn. J. Appl. Phys. 55,1102B2 (2016).75. D. V. Christensen, R. Dittmann, B. Linares-Barranco, A. Sebastian, M. le Gallo, A. Redaelli,S. Slesazeck, T. Mikolajick, S. Spiga, S. Menzel, I. Valov, G. Milano, C. Ricciardi, S. J. Liang,F. Miao, M. Lanza, T. J. Quill, S. T. Keene, A. Salleo, J. Grollier, D. Marković, A. Mizrahi, P. Yao,J. J. Yang, G. Indiveri, J. P. Strachan, S. Datta, E. Vianello, A. Valentian, J. Feldmann, X. Li,W. H. P. Pernice, H. Bhaskaran, S. Furber, E. Neftci, F. Scherr, W. Maass, S. Ramaswamy,J. Tapson, P. Panda, Y. Kim, G. Tanaka, S. Thorpe, C. Bartolozzi, T. A. Cleland, C. Posch,S. C. Liu, G. Panuccio, M. Mahmud, A. N. Mazumder, M. Hosseini, T. Mohsenin, E. Donati,S. Tolu, R. Galeazzi, M. E. Christensen, S. Holm, D. Ielmini, N. Pryds, 2022 Roadmap onneuromorphic computing and engineering. Neuromorphic Comput. Eng. 2, 022501 (2022).76. Y. Sandamirskaya, Rethinking computing hardware for robots. Sci. Robot. 7,eabq3909 (2022).77. Y. V. Pershin, V. A. Slipko, M. Di Ventra, Complex dynamics and scale invariance of one-dimensional memristive networks. Phys. Rev. E Stat. Nonlinear Soft Matter Phys. 87,022116 (2013).78. Z. Kuncic, O. Kavehei, R. Zhu, A. Loeffler, K. Fu, J. Hochstetter, M. Li, J. M. Shine, A. Diaz-Alvarez, A. Stieg, J. Gimzewski, T. Nakayama, Neuromorphic Information Processing withNanowire Networks, in Proceedings of the 2020 IEEE International Symposium on Circuits andSystems (ISCAS), Seville, Spain, 12 to 14 October 2020.79. Y. V. Pershin, V. A. Slipko, Dynamical attractors of memristors and their networks. Europhys.Lett. 125, 20002 (2019).80. Z. Wang, S. Joshi, S. E. Savel’ev, H. Jiang, R. Midya, P. Lin, M. Hu, N. Ge, J. P. Strachan, Z. Li,Q. Wu, M. Barnell, G. L. Li, H. L. Xin, R. S. Williams, Q. Xia, J. J. Yang, Memristors with diffusivedynamics as synaptic emulators for neuromorphic computing. Nat. Mater. 16,101–108 (2017).Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 13 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023https://arxiv.org/abs/2105.11233https://doi.org/10.4324/978141061240381. S. Gaba, P. Sheridan, J. Zhou, S. Choi, W. Lu, Stochastic memristive devices for computingand neuromorphic applications. Nanoscale 5, 5872–5878 (2013).82. S. K. Bose, S. Shirai, J. B. Mallinson, S. A. Brown, Synaptic dynamics in complex self-as-sembled nanoparticle networks. Faraday Discuss. 213, 471–485 (2019).Acknowledgments: We acknowledge use of the Artemis High Performance Computingresource at the Sydney Informatics Hub, a Core Research Facility of the University of Sydney.Funding: A.L. is supported by a Research Training Program scholarship from the University ofSydney. R.Z. is supported by a Postgraduate Research Excellence Award scholarship from theUniversity of Sydney. Author contributions: A.L., A.D.-A., and Z.K. conceived and designed thestudy. A.L. performed all simulations with assistance from R.Z., N.G., and Z.K. A.D.-A. performedall experiments, with support from T.N. A.L. and A.D.-A. analyzed and interpreted the data anddrafted the manuscript, with contributions from Z.K. and J.M.S. All authors critically reviewedthe manuscript. Z.K. supervised the project. Competing interests: Z.K. is with Emergentia Inc.The authors declare that they have no other competing interests. Data and materialsavailability: All data needed to evaluate the conclusions in the paper are present in the paperand/or the Supplementary Materials. Experimental data generated from physical networks inthis study have also been provided as source data file S1. Data files are also available via https://doi.org/10.5281/zenodo.7633957 and https://doi.org/10.5281/zenodo.7633957. Additionalsupporting materials are provided in the Supplementary Materials. Code used for allsimulations, experimental data analysis, and figure generation is available at the repositoryhttps://github.com/aloe8475/PhysicalReinforcementLearning or via https://doi.org/10.5281/zenodo.7633957.Submitted 18 December 2022Accepted 22 March 2023Published 21 April 202310.1126/sciadv.adg3289Loeffler et al., Sci. Adv. 9, eadg3289 (2023) 21 April 2023 14 of 14SC I ENCE ADVANCES | R E S EARCH ART I C L EDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023https://doi.org/10.5281/zenodo.7633957https://doi.org/10.5281/zenodo.7633957https://doi.org/10.5281/zenodo.7633957https://github.com/aloe8475/PhysicalReinforcementLearninghttps://doi.org/10.5281/zenodo.7633957https://doi.org/10.5281/zenodo.7633957https://doi.org/10.5281/zenodo.7633957Use of this article is subject to the Terms of serviceScience Advances (ISSN ) is published by the American Association for the Advancement of Science. 1200 New York Avenue NW,Washington, DC 20005. The title Science Advances is a registered trademark of AAAS.Copyright © 2023 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claimto original U.S. Government Works. Distributed under a Creative Commons Attribution NonCommercial License 4.0 (CC BY-NC).Neuromorphic learning, working memory, and metaplasticity in nanowire networksAlon Loeffler, Adrian Diaz-Alvarez, Ruomin Zhu, Natesh Ganesh, James M. Shine, Tomonobu Nakayama, and ZdenkaKuncicSci. Adv., 9 (16), eadg3289. DOI: 10.1126/sciadv.adg3289View the article onlinehttps://www.science.org/doi/10.1126/sciadv.adg3289Permissionshttps://www.science.org/help/reprints-and-permissionsDownloaded from https://www.science.org at National Institute for Materials Science on April 24, 2023https://www.science.org/content/page/terms-service INTRODUCTION RESULTS Task 1: Physical binary classification Task 2: Complex binary classification Task 3: Working memory Network connectivity DISCUSSION METHODS Experimental setup Simulation setup Implementation of supervised learning and PRL in classification and WM tasks Task 1 Task 2 Task 3 Supplementary Materials This PDF file includes: Other Supplementary Material for this manuscript includes the following: REFERENCES AND NOTES Acknowledgments