BasicOptimizer.scala:89 executed in 9.73 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 144369435742300
Reset training subject: 144369569321400
Adding measurement 346d622d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.486087574084142;dx=-0.03240283276426667
New Minimum: -1.486087574084142 > -1.5708938205854617
WOLFE (weak): th(2.154434690031884)=-1.5708938205854617; dx=-0.037876809568322255 evalInputDelta=0.08480624650131974
New Minimum: -1.5708938205854617 > -1.6568296605730666
WOLFE (weak): th(4.308869380063768)=-1.6568296605730666; dx=-0.0364830863074777 evalInputDelta=0.17074208648892464
New Minimum: -1.6568296605730666 > -2.0134554283223705
WOLFE (weak): th(12.926608140191302)=-2.0134554283223705; dx=-0.03515464958379738 evalInputDelta=0.5273678542382285
New Minimum: -2.0134554283223705 > -3.7977473508780273
WOLFE (weak): th(51.70643256076521)=-3.7977473508780273; dx=-0.03163368963373462 evalInputDelta=2.3116597767938853
New Minimum: -3.7977473508780273 > -15.220190712414402
WOLFE (weak): th(258.53216280382605)=-15.220190712414402; dx=-0.04186299908343019 evalInputDelta=13.73410313833026
WOLF (strong): th(1551.1929768229563)=-8.30783845224106; dx=3.162409174195784E-4 evalInputDelta=6.821750878156919
New Minimum: -15.220190712414402 > -16.274791410313643
END: th(904.8625698133912)=-16.274791410313643; dx=-0.001063910855900869 evalInputDelta=14.788703836229502
Fitness changed from -1.486087574084142 to -16.274791410313643
Iteration 1 complete. Error: -16.274791410313643 Total: 1.2841; Orientation: 0.0014; Line Search: 0.9276
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 13b6ee79 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-16.274791410313643;dx=-0.14619267655177767
New Minimum: -16.274791410313643 > -38.52969919070953
END: th(1949.4673101173673)=-38.52969919070953; dx=-0.003981594689883154 evalInputDelta=22.254907780395886
Fitness changed from -16.274791410313643 to -38.52969919070953
Iteration 2 complete. Error: -38.52969919070953 Total: 0.3793; Orientation: 0.0014; Line Search: 0.2511
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6ce75999 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-38.52969919070953;dx=-0.0553451649427785
New Minimum: -38.52969919070953 > -126.21758943511993
END: th(4200.000000000001)=-126.21758943511993; dx=-0.0038788367685273625 evalInputDelta=87.6878902444104
Fitness changed from -38.52969919070953 to -126.21758943511993
Iteration 3 complete. Error: -126.21758943511993 Total: 0.3490; Orientation: 0.0013; Line Search: 0.2277
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1c077e37 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.973e+02, gradient 2.760e-01, dot -0.212; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -126.21758943511993, -38.52969919070953, -16.274791410313643, -1.486087574084142
LBFGS Accumulation History: 3 points
Removed measurement 1c077e37 to history. Total: 3
th(0)=-126.21758943511993;dx=-0.07469047087765512
New Minimum: -126.21758943511993 > -407.51727340690724
END: th(9048.625698133914)=-407.51727340690724; dx=-0.00419727181313572 evalInputDelta=281.2996839717873
Fitness changed from -126.21758943511993 to -407.51727340690724
Iteration 4 complete. Error: -407.51727340690724 Total: 0.4318; Orientation: 0.0290; Line Search: 0.2814
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement e1cd4a8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.291e+03, gradient 2.836e-01, dot -0.194; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -407.51727340690724, -38.52969919070953, -16.274791410313643, -1.486087574084142
LBFGS Accumulation History: 3 points
Removed measurement e1cd4a8 to history. Total: 3
th(0)=-407.51727340690724;dx=-0.07534616185425133
New Minimum: -407.51727340690724 > -551.4948152521472
WOLF (strong): th(19494.673101173677)=-551.4948152521472; dx=2.536685195414385E-4 evalInputDelta=143.97754184523995
New Minimum: -551.4948152521472 > -747.3278647956331
END: th(9747.336550586839)=-747.3278647956331; dx=-0.007282078158028792 evalInputDelta=339.81059138872587
Fitness changed from -407.51727340690724 to -747.3278647956331
Iteration 5 complete. Error: -747.3278647956331 Total: 0.6964; Orientation: 0.0361; Line Search: 0.5012
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 703a99f7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.795e+04, gradient 3.507e-01, dot -0.242; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -747.3278647956331, -38.52969919070953, -16.274791410313643, -1.486087574084142
LBFGS Accumulation History: 3 points
Removed measurement 703a99f7 to history. Total: 3
th(0)=-747.3278647956331;dx=-0.11581816758904913
Armijo: th(21000.000000000007)=-694.675328840747; dx=0.006855778585878896 evalInputDelta=-52.65253595488605
New Minimum: -747.3278647956331 > -1037.6707690785165
END: th(10500.000000000004)=-1037.6707690785165; dx=-0.006086313908580521 evalInputDelta=290.3429042828834
Fitness changed from -747.3278647956331 to -1037.6707690785165
Iteration 6 complete. Error: -1037.6707690785165 Total: 0.6322; Orientation: 0.0337; Line Search: 0.4347
Adding measurement 4eb20fd2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.090e+05, gradient 3.843e-01, dot -0.236; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1037.6707690785165, -38.52969919070953, -16.274791410313643, -1.486087574084142
LBFGS Accumulation History: 3 points
Removed measurement 4eb20fd2 to history. Total: 3
th(0)=-1037.6707690785165;dx=-0.14194190012919677
Armijo: th(22621.564245334786)=-846.0918242129515; dx=0.011709171899086431 evalInputDelta=-191.57894486556506
New Minimum: -1037.6707690785165 > -1355.2967387255087
WOLF (strong): th(11310.782122667393)=-1355.2967387255087; dx=0.008462968365782541 evalInputDelta=317.6259696469922
New Minimum: -1355.2967387255087 > -1442.611217549621
END: th(3770.2607075557976)=-1442.611217549621; dx=-0.04058293472718094 evalInputDelta=404.94044847110445
Fitness changed from -1037.6707690785165 to -1442.611217549621
Iteration 7 complete. Error: -1442.611217549621 Total: 0.5952; Orientation: 0.0270; Line Search: 0.4578
Adding measurement 62244c56 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.956e+04, gradient 3.020e-01, dot 0.365; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1442.611217549621;dx=-3227.165936519451
Armijo: th(8122.780458822365)=-1411.9769828152298; dx=0.05795909303717152 evalInputDelta=-30.634234734391157
Armijo: th(4061.3902294111826)=-1412.3136602752238; dx=0.11429794814808955 evalInputDelta=-30.29755727439715
Armijo: th(1353.796743137061)=-1411.0591323541335; dx=0.334744474765965 evalInputDelta=-31.552085195487507
Armijo: th(338.44918578426524)=-1410.1940026109883; dx=1.337314396096926 evalInputDelta=-32.417214938632696
Armijo: th(67.68983715685305)=-1405.7405966251383; dx=7.065527070459533 evalInputDelta=-36.87062092448264
Armijo: th(11.281639526142174)=-1429.0827915134937; dx=39.045604726376844 evalInputDelta=-13.528426036127257
New Minimum: -1442.611217549621 > -1500.3948646939577
WOLF (strong): th(1.611662789448882)=-1500.3948646939577; dx=166.69881561255755 evalInputDelta=57.78364714433678
New Minimum: -1500.3948646939577 > -1658.4519890548536
END: th(0.20145784868111025)=-1658.4519890548536; dx=-505.13748736047035 evalInputDelta=215.84077150523262
Fitness changed from -1442.611217549621 to -1658.4519890548536
Iteration 8 complete. Error: -1658.4519890548536 Total: 1.1933; Orientation: 0.0271; Line Search: 1.0538
Adding measurement 4c9bee05 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.379e+04, gradient 3.124e-01, dot 0.336; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
th(0)=-1658.4519890548536;dx=-2405.4488626828274
Armijo: th(0.4340277777777779)=-1616.0580125725685; dx=26.636540086319513 evalInputDelta=-42.3939764822851
Armijo: th(0.21701388888888895)=-1651.909274143105; dx=-57.17486629040833 evalInputDelta=-6.54271491174859
New Minimum: -1658.4519890548536 > -1664.2155809870308
END: th(0.07233796296296298)=-1664.2155809870308; dx=-188.11473421721246 evalInputDelta=5.763591932177178
Fitness changed from -1658.4519890548536 to -1664.2155809870308
Iteration 9 complete. Error: -1664.2155809870308 Total: 0.6244; Orientation: 0.0339; Line Search: 0.4774
Adding measurement 721c41ac to history. Total: 5
Accepted: LBFGS Orientation magnitude: 2.140e+04, gradient 3.053e-01, dot 0.297; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
th(0)=-1664.2155809870308;dx=-2087.4310478790376
Armijo: th(0.15584741681364903)=-1652.609981211538; dx=-49.39415896050181 evalInputDelta=-11.605599775492692
Armijo: th(0.07792370840682451)=-1661.6989799599685; dx=-103.26470036642102 evalInputDelta=-2.516601027062279
New Minimum: -1664.2155809870308 > -1664.59206714438
END: th(0.025974569468941503)=-1664.59206714438; dx=-150.6771990091488 evalInputDelta=0.3764861573492908
Fitness changed from -1664.2155809870308 to -1664.59206714438
Iteration 10 complete. Error: -1664.59206714438 Total: 0.6490; Orientation: 0.0434; Line Search: 0.4923
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 12791111 to history. Total: 6
Accepted: LBFGS Orientation magnitude: 2.228e+04, gradient 3.079e-01, dot 0.314; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
th(0)=-1664.59206714438;dx=-2118.95705649037
Armijo: th(0.05596051352253062)=-1661.1521603995222; dx=-92.83946071146903 evalInputDelta=-3.439906744857808
Armijo: th(0.02798025676126531)=-1663.5764184431084; dx=-111.50139297405482 evalInputDelta=-1.0156487012716298
Armijo: th(0.009326752253755103)=-1664.49564433194; dx=-143.03699285925669 evalInputDelta=-0.0964228124400961
Armijo: th(0.0023316880634387757)=-1664.5832060909422; dx=-164.016796622277 evalInputDelta=-0.008861053437840383
New Minimum: -1664.59206714438 > -1664.5923675182462
END: th(4.663376126877551E-4)=-1664.5923675182462; dx=-144.0393802666111 evalInputDelta=3.003738661391253E-4
Fitness changed from -1664.59206714438 to -1664.5923675182462
Iteration 11 complete. Error: -1664.5923675182462 Total: 0.8913; Orientation: 0.0597; Line Search: 0.7037
Adding measurement 5e8ab48c to history. Total: 7
Accepted: LBFGS Orientation magnitude: 2.168e+04, gradient 3.103e-01, dot 0.305; [76d170c1-259d-4912-9a66-ca7a28ca9e11 = 1.000/1.000e+00]
th(0)=-1664.5923675182462;dx=-2058.4685942154647
Armijo: th(0.0010046939300411525)=-1664.588012091881; dx=-158.35593875835082 evalInputDelta=-0.004355426365236781
Armijo: th(5.023469650205762E-4)=-1664.5907905548438; dx=-140.97756098618214 evalInputDelta=-0.001576963402385445
Armijo: th(1.6744898834019208E-4)=-1664.59206714438; dx=-139.5355799726145 evalInputDelta=-3.003738661391253E-4
Armijo: th(4.186224708504802E-5)=-1664.591992050905; dx=-134.25835410719003 evalInputDelta=-3.7546734120041947E-4
Armijo: th(8.372449417009604E-6)=-1664.5922924247848; dx=-137.24440054075717 evalInputDelta=-7.509346141887363E-5
Armijo: th(1.3954082361682674E-6)=-1664.5921422378517; dx=-162.2954467428771 evalInputDelta=-2.2528039448843629E-4
Armijo: th(1.9934403373832392E-7)=-1664.5922924247848; dx=-160.18241714783204 evalInputDelta=-7.509346141887363E-5
Armijo: th(2.491800421729049E-8)=-1664.5921422378517; dx=-148.48797943722684 evalInputDelta=-2.2528039448843629E-4
Armijo: th(2.768667135254499E-9)=-1664.5922924247848; dx=-134.41946051075604 evalInputDelta=-7.509346141887363E-5
Armijo: th(2.768667135254499E-10)=-1664.5922924247848; dx=-155.16320376095462 evalInputDelta=-7.509346141887363E-5
MIN ALPHA (2.5169701229586355E-11): th(0.0)=-1664.5923675182462
Fitness changed from -1664.5923675182462 to -1664.5923675182462
Static Iteration Total: 1.5464; Orientation: 0.0542; Line Search: 1.3845
Iteration 12 failed. Error: -1664.5923675182462
Previous Error: 0.0 -> -1664.5923675182462
Retrying iteration 12
Reset training subject: 144378708440700
Adding measurement 4d8ca3ed to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-1664.5923675182462;dx=-0.09053854686213228
END: th(2.1544346900318838E-10)=-1664.5923675182462; dx=-0.05065472279515204 evalInputDelta=0.0
Fitness changed from -1664.5923675182462 to -1664.5923675182462
Static Iteration Total: 0.4584; Orientation: 0.0015; Line Search: 0.2359
Iteration 13 failed. Error: -1664.5923675182462
Previous Error: 0.0 -> -1664.5923675182462
Optimization terminated 13
Final threshold in iteration 13: -1664.5923675182462 (> -Infinity) after 9.731s (< 5400.000s)
Returns
-1664.5923675182462