BasicOptimizer.scala:89 executed in 136.02 seconds (1.594 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 335429525520000
Reset training subject: 335432300121000
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -113.13851203147452 > -113.24166720644783
th(0)=-113.24166720644783;dx=-2.8377877303079466E-5
New Minimum: -113.24166720644783 > -113.24171830912493
END: th(2.154434690031884)=-113.24171830912493; dx=-2.1515139593507394E-5 evalInputDelta=5.1102677105063776E-5
Fitness changed from -113.13851203147452 to -113.24171830912493
Iteration 1 complete. Error: -113.24171830912493 Total: 13.1171; Orientation: 0.0430; Line Search: 5.2797
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 20c2ea30 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-113.24171830912493;dx=-2.1488431062065743E-5
New Minimum: -113.24171830912493 > -113.2418332900641
WOLFE (weak): th(4.641588833612779)=-113.2418332900641; dx=-2.1465593184408492E-5 evalInputDelta=1.1498093915918162E-4
New Minimum: -113.2418332900641 > -113.24194401233962
WOLFE (weak): th(9.283177667225559)=-113.24194401233962; dx=-2.1456602383100716E-5 evalInputDelta=2.2570321468151633E-4
New Minimum: -113.24194401233962 > -113.24240393447909
WOLFE (weak): th(27.849533001676676)=-113.24240393447909; dx=-2.14553432056871E-5 evalInputDelta=6.856253541513979E-4
New Minimum: -113.24240393447909 > -113.24444375182472
WOLFE (weak): th(111.3981320067067)=-113.24444375182472; dx=-2.1436554857818328E-5 evalInputDelta=0.002725442699784253
New Minimum: -113.24444375182472 > -113.25532783458378
WOLFE (weak): th(556.9906600335335)=-113.25532783458378; dx=-2.14354079868962E-5 evalInputDelta=0.01360952545884686
New Minimum: -113.25532783458378 > -113.32333392346511
WOLFE (weak): th(3341.9439602012008)=-113.32333392346511; dx=-2.1418655379665492E-5 evalInputDelta=0.08161561434017983
New Minimum: -113.32333392346511 > -113.81070109434091
WOLFE (weak): th(23393.607721408407)=-113.81070109434091; dx=-2.1236970413170317E-5 evalInputDelta=0.5689827852159794
New Minimum: -113.81070109434091 > -117.62588196662857
WOLFE (weak): th(187148.86177126726)=-117.62588196662857; dx=-1.9630771667980363E-5 evalInputDelta=4.384163657503635
New Minimum: -117.62588196662857 > -137.69624313047737
END: th(1684339.7559414054)=-137.69624313047737; dx=-5.625604411963028E-6 evalInputDelta=24.454524821352436
Fitness changed from -113.24171830912493 to -137.69624313047737
Iteration 2 complete. Error: -137.69624313047737 Total: 29.6803; Orientation: 0.0634; Line Search: 26.4498
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2a9368ff to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-137.69624313047737;dx=-1.0595065472937728E-5
New Minimum: -137.69624313047737 > -156.10336324397187
END: th(3628800.0000000005)=-156.10336324397187; dx=-8.906672841014716E-7 evalInputDelta=18.4071201134945
Fitness changed from -137.69624313047737 to -156.10336324397187
Iteration 3 complete. Error: -156.10336324397187 Total: 8.3189; Orientation: 0.0623; Line Search: 5.3183
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 69d2cd03 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-156.10336324397187;dx=-1.0740994518598332E-5
Armijo: th(7818012.603187701)=-152.87362174090077; dx=3.2916504255624496E-6 evalInputDelta=-3.2297415030710965
New Minimum: -156.10336324397187 > -167.2510072880943
WOLF (strong): th(3909006.3015938504)=-167.2510072880943; dx=1.9338909142630733E-6 evalInputDelta=11.147644044122444
END: th(1303002.1005312835)=-166.64013441902728; dx=-4.0396619414444555E-6 evalInputDelta=10.536771175055407
Fitness changed from -156.10336324397187 to -167.2510072880943
Iteration 4 complete. Error: -167.2510072880943 Total: 14.3673; Orientation: 0.0633; Line Search: 10.6643
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement f0e443 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.945e+04, gradient 3.354e-03, dot -0.518; [07b06d01-bfc4-4001-b5bc-e88faded6f86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -167.2510072880943, -156.10336324397187, -137.69624313047737, -113.24171830912493
LBFGS Accumulation History: 3 points
Removed measurement f0e443 to history. Total: 3
th(0)=-167.2510072880943;dx=-1.1231453621416039E-5
New Minimum: -167.2510072880943 > -183.06794598161656
WOLF (strong): th(2807232.926569009)=-183.06794598161656; dx=4.3673175892960293E-7 evalInputDelta=15.816938693522246
END: th(1403616.4632845046)=-180.07617878535027; dx=-4.868461651397216E-6 evalInputDelta=12.825171497255951
Fitness changed from -167.2510072880943 to -183.06794598161656
Iteration 5 complete. Error: -183.06794598161656 Total: 12.4657; Orientation: 1.2464; Line Search: 7.8231
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7fbcb0ba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.693e+04, gradient 2.568e-03, dot -0.532; [07b06d01-bfc4-4001-b5bc-e88faded6f86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -183.06794598161656, -156.10336324397187, -137.69624313047737, -113.24171830912493
LBFGS Accumulation History: 3 points
Removed measurement 7fbcb0ba to history. Total: 3
th(0)=-183.06794598161656;dx=-6.583357560369026E-6
New Minimum: -183.06794598161656 > -192.65030804474077
WOLF (strong): th(3024000.0000000005)=-192.65030804474077; dx=1.7669468887610345E-7 evalInputDelta=9.582362063124208
END: th(1512000.0000000002)=-190.8619576393137; dx=-2.7943878221713285E-6 evalInputDelta=7.794011657697126
Fitness changed from -183.06794598161656 to -192.65030804474077
Iteration 6 complete. Error: -192.65030804474077 Total: 11.7413; Orientation: 1.1732; Line Search: 7.5386
Adding measurement 221a42bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.276e+04, gradient 2.447e-03, dot -0.542; [07b06d01-bfc4-4001-b5bc-e88faded6f86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -192.65030804474077, -156.10336324397187, -137.69624313047737, -113.24171830912493
LBFGS Accumulation History: 3 points
Removed measurement 221a42bb to history. Total: 3
th(0)=-192.65030804474077;dx=-5.976927496346517E-6
New Minimum: -192.65030804474077 > -199.90563707550848
WOLF (strong): th(3257505.251328209)=-199.90563707550848; dx=1.026977976645879E-6 evalInputDelta=7.255329030767712
END: th(1628752.6256641045)=-199.6773835138207; dx=-1.921115797574643E-6 evalInputDelta=7.027075469079932
Fitness changed from -192.65030804474077 to -199.90563707550848
Iteration 7 complete. Error: -199.90563707550848 Total: 11.2942; Orientation: 1.0212; Line Search: 7.6050
Adding measurement 8100a87 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.815e+04, gradient 2.559e-03, dot -0.475; [07b06d01-bfc4-4001-b5bc-e88faded6f86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -199.90563707550848, -156.10336324397187, -137.69624313047737, -113.24171830912493
LBFGS Accumulation History: 3 points
Removed measurement 8100a87 to history. Total: 3
th(0)=-199.90563707550848;dx=-6.546010617887296E-6
New Minimum: -199.90563707550848 > -205.94650888485265
WOLF (strong): th(3509041.158211262)=-205.94650888485265; dx=2.2474221721585674E-6 evalInputDelta=6.040871809344168
New Minimum: -205.94650888485265 > -207.5617759975197
END: th(1754520.579105631)=-207.5617759975197; dx=-1.4220139011098725E-6 evalInputDelta=7.656138922011223
Fitness changed from -199.90563707550848 to -207.5617759975197
Iteration 8 complete. Error: -207.5617759975197 Total: 11.4568; Orientation: 1.0259; Line Search: 7.9215
Adding measurement 4e28249d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.442e+04, gradient 1.678e-03, dot -0.688; [07b06d01-bfc4-4001-b5bc-e88faded6f86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -207.5617759975197, -156.10336324397187, -137.69624313047737, -113.24171830912493
LBFGS Accumulation History: 3 points
Removed measurement 4e28249d to history. Total: 3
th(0)=-207.5617759975197;dx=-2.808962550218041E-6
New Minimum: -207.5617759975197 > -214.46558463720265
END: th(3780000.0000000014)=-214.46558463720265; dx=-8.839678956693019E-7 evalInputDelta=6.90380863968295
Fitness changed from -207.5617759975197 to -214.46558463720265
Iteration 9 complete. Error: -214.46558463720265 Total: 8.8582; Orientation: 1.0211; Line Search: 5.4061
Adding measurement 15688005 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.572e+04, gradient 1.981e-03, dot -0.546; [07b06d01-bfc4-4001-b5bc-e88faded6f86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -214.46558463720265, -156.10336324397187, -137.69624313047737, -113.24171830912493
LBFGS Accumulation History: 3 points
Removed measurement 15688005 to history. Total: 3
th(0)=-214.46558463720265;dx=-3.911993444771983E-6
Armijo: th(8143763.128320524)=-210.5718112442043; dx=1.866609862958264E-6 evalInputDelta=-3.893773392998355
New Minimum: -214.46558463720265 > -218.36663870794834
WOLF (strong): th(4071881.564160262)=-218.36663870794834; dx=8.481111987836756E-7 evalInputDelta=3.901054070745687
New Minimum: -218.36663870794834 > -218.46325796945666
END: th(1357293.8547200873)=-218.46325796945666; dx=-1.603129947208469E-6 evalInputDelta=3.99767333225401
Fitness changed from -214.46558463720265 to -218.46325796945666
Iteration 10 complete. Error: -218.46325796945666 Total: 14.1122; Orientation: 1.0544; Line Search: 10.3569
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -218.46325796945666 (> -Infinity) after 136.021s (< 5400.000s)
Returns
-218.46325796945666