BasicOptimizer.scala:89 executed in 61.44 seconds (0.962 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 148841846894200
Reset training subject: 148842775301800
Adding measurement 649b3415 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-75.90327803980207;dx=-0.015024148736620321
New Minimum: -75.90327803980207 > -75.9343910114597
WOLFE (weak): th(2.154434690031884)=-75.9343910114597; dx=-0.014957637132213732 evalInputDelta=0.031112971657620392
New Minimum: -75.9343910114597 > -75.96550409560908
WOLFE (weak): th(4.308869380063768)=-75.96550409560908; dx=-0.014928640523587767 evalInputDelta=0.06222605580700247
New Minimum: -75.96550409560908 > -76.09011156599128
WOLFE (weak): th(12.926608140191302)=-76.09011156599128; dx=-0.01492823723761552 evalInputDelta=0.18683352618920424
New Minimum: -76.09011156599128 > -76.65016027584352
WOLFE (weak): th(51.70643256076521)=-76.65016027584352; dx=-0.01491653715482839 evalInputDelta=0.7468822360414435
New Minimum: -76.65016027584352 > -79.62620191400566
WOLFE (weak): th(258.53216280382605)=-79.62620191400566; dx=-0.014615894927222754 evalInputDelta=3.7229238742035875
New Minimum: -79.62620191400566 > -97.50069611129707
END: th(1551.1929768229563)=-97.50069611129707; dx=-0.013152143547732071 evalInputDelta=21.597418071495
Fitness changed from -75.90327803980207 to -97.50069611129707
Iteration 1 complete. Error: -97.50069611129707 Total: 9.0563; Orientation: 0.0155; Line Search: 6.4211
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 41119691 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-97.50069611129707;dx=-0.014740050559877607
New Minimum: -97.50069611129707 > -142.2375736004731
END: th(3341.943960201201)=-142.2375736004731; dx=-0.011342228533735017 evalInputDelta=44.736877489176024
Fitness changed from -97.50069611129707 to -142.2375736004731
Iteration 2 complete. Error: -142.2375736004731 Total: 2.5894; Orientation: 0.0149; Line Search: 1.6819
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 277f07fa to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-142.2375736004731;dx=-0.014688504071785267
New Minimum: -142.2375736004731 > -227.20771918279976
END: th(7200.000000000001)=-227.20771918279976; dx=-0.009717757965670403 evalInputDelta=84.97014558232667
Fitness changed from -142.2375736004731 to -227.20771918279976
Iteration 3 complete. Error: -227.20771918279976 Total: 2.6887; Orientation: 0.0213; Line Search: 1.6955
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement f94d0ff to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.690e+04, gradient 1.328e-01, dot -0.488; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -227.20771918279976, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement f94d0ff to history. Total: 3
th(0)=-227.20771918279976;dx=-0.017622991500542216
New Minimum: -227.20771918279976 > -375.31172460369527
END: th(15511.929768229566)=-375.31172460369527; dx=-0.005707180363332192 evalInputDelta=148.1040054208955
Fitness changed from -227.20771918279976 to -375.31172460369527
Iteration 4 complete. Error: -375.31172460369527 Total: 3.1890; Orientation: 0.3010; Line Search: 1.9233
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 61fb80cc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.478e+04, gradient 1.581e-01, dot -0.254; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -375.31172460369527, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 61fb80cc to history. Total: 3
th(0)=-375.31172460369527;dx=-0.02477073187457203
New Minimum: -375.31172460369527 > -633.589994594296
END: th(33419.43960201202)=-633.589994594296; dx=-0.0015219815611952582 evalInputDelta=258.2782699906007
Fitness changed from -375.31172460369527 to -633.589994594296
Iteration 5 complete. Error: -633.589994594296 Total: 2.8219; Orientation: 0.2824; Line Search: 1.6596
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 44afb47f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.609e+04, gradient 1.819e-01, dot -0.125; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -633.589994594296, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 44afb47f to history. Total: 3
th(0)=-633.589994594296;dx=-0.033026491381187795
New Minimum: -633.589994594296 > -782.757465630319
WOLF (strong): th(72000.00000000003)=-782.757465630319; dx=0.0035269333329034785 evalInputDelta=149.16747103602302
New Minimum: -782.757465630319 > -965.8123523749321
END: th(36000.000000000015)=-965.8123523749321; dx=-8.175331309311442E-6 evalInputDelta=332.2223577806361
Fitness changed from -633.589994594296 to -965.8123523749321
Iteration 6 complete. Error: -965.8123523749321 Total: 3.7958; Orientation: 0.2875; Line Search: 2.5729
Adding measurement 6c33d31b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.345e+04, gradient 1.919e-01, dot -0.130; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -965.8123523749321, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 6c33d31b to history. Total: 3
th(0)=-965.8123523749321;dx=-0.03675676493524585
New Minimum: -965.8123523749321 > -1013.3839043028066
WOLF (strong): th(77559.64884114785)=-1013.3839043028066; dx=0.0046386792026036645 evalInputDelta=47.571551927874566
New Minimum: -1013.3839043028066 > -1328.0709224284674
WOLF (strong): th(38779.824420573925)=-1328.0709224284674; dx=0.0019616224288084376 evalInputDelta=362.2585700535353
END: th(12926.608140191309)=-1309.9086800231535; dx=-0.011779060472437312 evalInputDelta=344.0963276482214
Fitness changed from -965.8123523749321 to -1328.0709224284674
Iteration 7 complete. Error: -1328.0709224284674 Total: 4.7395; Orientation: 0.2922; Line Search: 3.6169
Adding measurement 37398179 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.469e+04, gradient 1.995e-01, dot -0.144; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1328.0709224284674, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 37398179 to history. Total: 3
th(0)=-1328.0709224284674;dx=-0.03973457748985474
New Minimum: -1328.0709224284674 > -1716.9090686463276
WOLF (strong): th(27849.53300167669)=-1716.9090686463276; dx=2.3441476758325006E-4 evalInputDelta=388.8381462178602
END: th(13924.766500838345)=-1690.6665105809602; dx=-0.01044787639002307 evalInputDelta=362.59558815249284
Fitness changed from -1328.0709224284674 to -1716.9090686463276
Iteration 8 complete. Error: -1716.9090686463276 Total: 3.7167; Orientation: 0.2789; Line Search: 2.5947
Adding measurement 614d4041 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.131e+05, gradient 1.854e-01, dot 0.220; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1716.9090686463276;dx=-4634.86934398754
Armijo: th(30000.00000000002)=-192.2693118551424; dx=0.0077840689289597805 evalInputDelta=-1524.6397567911852
Armijo: th(15000.00000000001)=-192.30358282868784; dx=0.01555102187433209 evalInputDelta=-1524.6054858176396
Armijo: th(5000.000000000003)=-192.268702357534; dx=0.04663702088314301 evalInputDelta=-1524.6403662887935
Armijo: th(1250.0000000000007)=-192.00209552892906; dx=0.18654796119738662 evalInputDelta=-1524.9069731173986
Armijo: th(250.00000000000014)=-192.61768926749693; dx=0.9285323083977002 evalInputDelta=-1524.2913793788307
Armijo: th(41.66666666666669)=-195.30229973364368; dx=5.795299305298005 evalInputDelta=-1521.606768912684
Armijo: th(5.952380952380956)=-216.62476557979238; dx=44.75858476029775 evalInputDelta=-1500.2843030665351
Armijo: th(0.7440476190476195)=-556.4088537217933; dx=542.4835765053049 evalInputDelta=-1160.5002149245342
Armijo: th(0.08267195767195773)=-1691.4422100680827; dx=1355.8922054021311 evalInputDelta=-25.46685857824491
New Minimum: -1716.9090686463276 > -1746.6096444254508
END: th(0.008267195767195774)=-1746.6096444254508; dx=-2093.4350563398693 evalInputDelta=29.700575779123255
Fitness changed from -1716.9090686463276 to -1746.6096444254508
Iteration 9 complete. Error: -1746.6096444254508 Total: 10.9223; Orientation: 0.2776; Line Search: 9.8701
Adding measurement 35f849ce to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.033e+06, gradient 1.850e-01, dot -0.174; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.6096444254508, -1716.9090686463276, -142.2375736004731, -97.50069611129707, -75.90327803980207
Rejected: LBFGS Orientation magnitude: 9.907e+05, gradient 1.850e-01, dot -0.210; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.6096444254508, -1716.9090686463276, -142.2375736004731, -97.50069611129707
LBFGS Accumulation History: 3 points
Removed measurement 35f849ce to history. Total: 4
Removed measurement 614d4041 to history. Total: 3
th(0)=-1746.6096444254508;dx=-0.03414682621090322
New Minimum: -1746.6096444254508 > -1746.6103600975232
END: th(0.01781113335013133)=-1746.6103600975232; dx=-0.029056441260156426 evalInputDelta=7.156720723742183E-4
Fitness changed from -1746.6096444254508 to -1746.6103600975232
Iteration 10 complete. Error: -1746.6103600975232 Total: 3.2103; Orientation: 0.6178; Line Search: 1.7490
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 31439e38 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.833e+09, gradient 1.851e-01, dot -0.174; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.6103600975232, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 31439e38 to history. Total: 3
th(0)=-1746.6103600975232;dx=-0.03419129423885621
New Minimum: -1746.6103600975232 > -1746.6117914407885
END: th(0.03837292355830674)=-1746.6117914407885; dx=-0.029002688939330445 evalInputDelta=0.00143134326526706
Fitness changed from -1746.6103600975232 to -1746.6117914407885
Iteration 11 complete. Error: -1746.6117914407885 Total: 3.1619; Orientation: 0.2784; Line Search: 1.9855
Adding measurement 25479e5f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.778e+10, gradient 1.850e-01, dot -0.174; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.6117914407885, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 25479e5f to history. Total: 3
th(0)=-1746.6117914407885;dx=-0.03412053593626328
New Minimum: -1746.6117914407885 > -1746.6148688248363
END: th(0.08267195767195774)=-1746.6148688248363; dx=-0.029050019236804135 evalInputDelta=0.0030773840478559578
Fitness changed from -1746.6117914407885 to -1746.6148688248363
Iteration 12 complete. Error: -1746.6148688248363 Total: 2.8990; Orientation: 0.2816; Line Search: 1.7874
Adding measurement 428b96ec to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.369e+09, gradient 1.850e-01, dot -0.174; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.6148688248363, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 428b96ec to history. Total: 3
th(0)=-1746.6148688248363;dx=-0.03415305947859035
New Minimum: -1746.6148688248363 > -1746.621309843665
END: th(0.1781113335013133)=-1746.621309843665; dx=-0.028969875461158645 evalInputDelta=0.006441018828581946
Fitness changed from -1746.6148688248363 to -1746.621309843665
Iteration 13 complete. Error: -1746.621309843665 Total: 2.8266; Orientation: 0.2995; Line Search: 1.6849
Adding measurement 55c6a3ec to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.845e+08, gradient 1.850e-01, dot -0.174; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.621309843665, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 55c6a3ec to history. Total: 3
th(0)=-1746.621309843665;dx=-0.03414556951751426
New Minimum: -1746.621309843665 > -1746.6353368691473
END: th(0.38372923558306743)=-1746.6353368691473; dx=-0.028993344443667475 evalInputDelta=0.014027025482391764
Fitness changed from -1746.621309843665 to -1746.6353368691473
Iteration 14 complete. Error: -1746.6353368691473 Total: 2.7289; Orientation: 0.2692; Line Search: 1.6716
Adding measurement 5ef67197 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.727e+11, gradient 1.850e-01, dot -0.174; [27046425-edf1-45e9-9837-7e25d91cabfd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.6353368691473, -142.2375736004731, -97.50069611129707, -75.90327803980207
LBFGS Accumulation History: 3 points
Removed measurement 5ef67197 to history. Total: 3
th(0)=-1746.6353368691473;dx=-0.03416207739978628
New Minimum: -1746.6353368691473 > -1746.6655375314417
END: th(0.8267195767195776)=-1746.6655375314417; dx=-0.029037002540431872 evalInputDelta=0.03020066229441909
Fitness changed from -1746.6353368691473 to -1746.6655375314417
Iteration 15 complete. Error: -1746.6655375314417 Total: 2.9955; Orientation: 0.2849; Line Search: 1.7660
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1746.6655375314417 (> -Infinity) after 61.442s (< 5400.000s)
Returns
-1746.6655375314417