BasicOptimizer.scala:89 executed in 46.21 seconds (1.359 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 220218964545600
Reset training subject: 220219564592300
Adding measurement 76b5ec48 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-58.764688577650425;dx=-0.011289144630414503
New Minimum: -58.764688577650425 > -58.781294366714185
WOLFE (weak): th(2.154434690031884)=-58.781294366714185; dx=-0.011284293601466825 evalInputDelta=0.016605789063760312
New Minimum: -58.781294366714185 > -58.797888208976666
WOLFE (weak): th(4.308869380063768)=-58.797888208976666; dx=-0.011278433686004495 evalInputDelta=0.03319963132624082
New Minimum: -58.797888208976666 > -58.864288251738714
WOLFE (weak): th(12.926608140191302)=-58.864288251738714; dx=-0.011262650561105463 evalInputDelta=0.09959967408828874
New Minimum: -58.864288251738714 > -59.16270919313544
WOLFE (weak): th(51.70643256076521)=-59.16270919313544; dx=-0.011215082415357092 evalInputDelta=0.39802061548501655
New Minimum: -59.16270919313544 > -60.709337459776556
WOLFE (weak): th(258.53216280382605)=-60.709337459776556; dx=-0.010942134783628977 evalInputDelta=1.9446488821261312
New Minimum: -60.709337459776556 > -69.49449586281283
END: th(1551.1929768229563)=-69.49449586281283; dx=-0.009799741953935006 evalInputDelta=10.729807285162401
Fitness changed from -58.764688577650425 to -69.49449586281283
Iteration 1 complete. Error: -69.49449586281283 Total: 6.0830; Orientation: 0.0166; Line Search: 4.3130
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 33e22406 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-69.49449586281283;dx=-0.012691453769441993
New Minimum: -69.49449586281283 > -94.7431465354633
END: th(3341.943960201201)=-94.7431465354633; dx=-0.008543225900088162 evalInputDelta=25.24865067265047
Fitness changed from -69.49449586281283 to -94.7431465354633
Iteration 2 complete. Error: -94.7431465354633 Total: 1.7992; Orientation: 0.0164; Line Search: 1.2145
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 65838c35 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-94.7431465354633;dx=-0.013666538687548946
New Minimum: -94.7431465354633 > -127.6741436572234
END: th(7200.000000000001)=-127.6741436572234; dx=-0.004365209635639163 evalInputDelta=32.9309971217601
Fitness changed from -94.7431465354633 to -127.6741436572234
Iteration 3 complete. Error: -127.6741436572234 Total: 2.1102; Orientation: 0.0161; Line Search: 1.5060
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 613c0b53 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.259e+04, gradient 1.288e-01, dot -0.243; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -127.6741436572234, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 613c0b53 to history. Total: 3
th(0)=-127.6741436572234;dx=-0.016566982766887683
New Minimum: -127.6741436572234 > -135.6058705413516
END: th(15511.929768229566)=-135.6058705413516; dx=-0.0015639968629149125 evalInputDelta=7.93172688412821
Fitness changed from -127.6741436572234 to -135.6058705413516
Iteration 4 complete. Error: -135.6058705413516 Total: 2.0683; Orientation: 0.2922; Line Search: 1.1750
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 59d884eb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.016e+04, gradient 1.161e-01, dot -0.182; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -135.6058705413516, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 59d884eb to history. Total: 3
th(0)=-135.6058705413516;dx=-0.01347031247809662
New Minimum: -135.6058705413516 > -163.61799473472684
END: th(33419.43960201202)=-163.61799473472684; dx=-0.0010645554663404405 evalInputDelta=28.01212419337523
Fitness changed from -135.6058705413516 to -163.61799473472684
Iteration 5 complete. Error: -163.61799473472684 Total: 2.0221; Orientation: 0.2953; Line Search: 1.1619
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7d197f6d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.921e+03, gradient 1.092e-01, dot -0.113; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -163.61799473472684, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 7d197f6d to history. Total: 3
th(0)=-163.61799473472684;dx=-0.011897642852951945
Armijo: th(72000.00000000003)=-151.0697528072736; dx=-3.832490492124668E-4 evalInputDelta=-12.548241927453233
New Minimum: -163.61799473472684 > -222.42988592695298
END: th(36000.000000000015)=-222.42988592695298; dx=-0.0019042229502275055 evalInputDelta=58.81189119222614
Fitness changed from -163.61799473472684 to -222.42988592695298
Iteration 6 complete. Error: -222.42988592695298 Total: 2.7818; Orientation: 0.2899; Line Search: 1.9010
Adding measurement 532f3fc2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.022e+03, gradient 1.077e-01, dot -0.112; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -222.42988592695298, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 532f3fc2 to history. Total: 3
th(0)=-222.42988592695298;dx=-0.011557077825836391
Armijo: th(77559.64884114785)=-184.3209852989375; dx=-3.7733786959974003E-4 evalInputDelta=-38.108900628015476
New Minimum: -222.42988592695298 > -279.48421268358317
END: th(38779.824420573925)=-279.48421268358317; dx=-0.002281760993490542 evalInputDelta=57.05432675663019
Fitness changed from -222.42988592695298 to -279.48421268358317
Iteration 7 complete. Error: -279.48421268358317 Total: 2.8057; Orientation: 0.3298; Line Search: 1.8677
Adding measurement 16b9dc3f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.373e+04, gradient 1.047e-01, dot -0.122; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -279.48421268358317, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 16b9dc3f to history. Total: 3
th(0)=-279.48421268358317;dx=-0.010900936240577632
Armijo: th(83548.59900503006)=-242.64184462957184; dx=-4.3922234261988463E-4 evalInputDelta=-36.84236805401133
New Minimum: -279.48421268358317 > -345.71839008297803
END: th(41774.29950251503)=-345.71839008297803; dx=-0.0025275250481744463 evalInputDelta=66.23417739939487
Fitness changed from -279.48421268358317 to -345.71839008297803
Iteration 8 complete. Error: -345.71839008297803 Total: 2.9068; Orientation: 0.2743; Line Search: 1.8577
Adding measurement e937028 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.643e+04, gradient 1.011e-01, dot -0.145; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -345.71839008297803, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement e937028 to history. Total: 3
th(0)=-345.71839008297803;dx=-0.010190766489923052
Armijo: th(90000.00000000004)=-292.0763417388741; dx=-4.8064932114874067E-4 evalInputDelta=-53.642048344103955
New Minimum: -345.71839008297803 > -416.86256851979084
END: th(45000.00000000002)=-416.86256851979084; dx=-0.002739653959799199 evalInputDelta=71.14417843681281
Fitness changed from -345.71839008297803 to -416.86256851979084
Iteration 9 complete. Error: -416.86256851979084 Total: 2.9669; Orientation: 0.4044; Line Search: 1.9224
Adding measurement 6cbfff35 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.691e+04, gradient 1.018e-01, dot -0.169; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -416.86256851979084, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 6cbfff35 to history. Total: 3
th(0)=-416.86256851979084;dx=-0.010345743419858599
Armijo: th(96949.56105143482)=-335.2220401451713; dx=-4.326138942795975E-4 evalInputDelta=-81.64052837461952
New Minimum: -416.86256851979084 > -480.27423039744326
END: th(48474.78052571741)=-480.27423039744326; dx=-0.0026319584624048733 evalInputDelta=63.41166187765242
Fitness changed from -416.86256851979084 to -480.27423039744326
Iteration 10 complete. Error: -480.27423039744326 Total: 2.7885; Orientation: 0.3708; Line Search: 1.7898
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4851ae11 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.807e+05, gradient 1.024e-01, dot -0.174; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -480.27423039744326, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 4851ae11 to history. Total: 3
th(0)=-480.27423039744326;dx=-0.010484252129365827
Armijo: th(104435.74875628759)=-383.8748197549769; dx=-4.3667565737558737E-4 evalInputDelta=-96.39941064246636
New Minimum: -480.27423039744326 > -545.5325735962584
END: th(52217.874378143795)=-545.5325735962584; dx=-0.002720214110519572 evalInputDelta=65.25834319881517
Fitness changed from -480.27423039744326 to -545.5325735962584
Iteration 11 complete. Error: -545.5325735962584 Total: 2.7083; Orientation: 0.2945; Line Search: 1.8185
Adding measurement 34e3f0 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.479e+04, gradient 1.017e-01, dot 0.204; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-545.5325735962584;dx=-1758.2070645803446
Armijo: th(112500.00000000007)=611.0011872929998; dx=0.004956593495706488 evalInputDelta=-1156.5337608892582
Armijo: th(56250.00000000004)=610.6999270716917; dx=0.009936330204230714 evalInputDelta=-1156.2325006679503
Armijo: th(18750.00000000001)=609.8955157703258; dx=0.029757588286412753 evalInputDelta=-1155.4280893665841
Armijo: th(4687.500000000003)=607.3133783082277; dx=0.11860643403121163 evalInputDelta=-1152.8459519044861
Armijo: th(937.5000000000006)=589.8042525809371; dx=0.5679740624192076 evalInputDelta=-1135.3368261771957
Armijo: th(156.25000000000009)=547.2916478719219; dx=3.123499411574137 evalInputDelta=-1092.8242214681804
Armijo: th(22.321428571428584)=480.64256017896963; dx=20.400849008840652 evalInputDelta=-1026.175133775228
Armijo: th(2.790178571428573)=230.3005398232201; dx=112.94785888356354 evalInputDelta=-775.8331134194785
Armijo: th(0.31001984126984145)=-324.6061291770803; dx=199.6730455169636 evalInputDelta=-220.92644441917815
New Minimum: -545.5325735962584 > -550.5515849755345
END: th(0.031001984126984145)=-550.5515849755345; dx=-584.5752497447259 evalInputDelta=5.019011379276094
Fitness changed from -545.5325735962584 to -550.5515849755345
Iteration 12 complete. Error: -550.5515849755345 Total: 7.8213; Orientation: 0.2792; Line Search: 6.7324
Adding measurement 4c443c58 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.080e+06, gradient 1.001e-01, dot -0.168; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -550.5515849755345, -545.5325735962584, -94.7431465354633, -69.49449586281283, -58.764688577650425
Rejected: LBFGS Orientation magnitude: 1.208e+06, gradient 1.001e-01, dot -0.200; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -550.5515849755345, -545.5325735962584, -94.7431465354633, -69.49449586281283
LBFGS Accumulation History: 3 points
Removed measurement 4c443c58 to history. Total: 4
Removed measurement 34e3f0 to history. Total: 3
th(0)=-550.5515849755345;dx=-0.009982778398459501
New Minimum: -550.5515849755345 > -550.5519230138462
END: th(0.06679175006299247)=-550.5519230138462; dx=-0.008470485359327214 evalInputDelta=3.3803831172463106E-4
Fitness changed from -550.5515849755345 to -550.5519230138462
Iteration 13 complete. Error: -550.5519230138462 Total: 2.3807; Orientation: 0.6366; Line Search: 1.2076
Adding measurement 1b11e65b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.015e+05, gradient 1.000e-01, dot -0.175; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -550.5519230138462, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 1b11e65b to history. Total: 3
th(0)=-550.5519230138462;dx=-0.009972282904756213
New Minimum: -550.5519230138462 > -550.5526892333164
END: th(0.14389846334365022)=-550.5526892333164; dx=-0.008450010719036621 evalInputDelta=7.66219470165197E-4
Fitness changed from -550.5519230138462 to -550.5526892333164
Iteration 14 complete. Error: -550.5526892333164 Total: 2.8813; Orientation: 1.0222; Line Search: 1.3005
Adding measurement 33eeb708 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.013e+05, gradient 1.000e-01, dot -0.175; [bb5e6d8b-1551-4eb2-8fca-75b59d10f194 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -550.5526892333164, -94.7431465354633, -69.49449586281283, -58.764688577650425
LBFGS Accumulation History: 3 points
Removed measurement 33eeb708 to history. Total: 3
th(0)=-550.5526892333164;dx=-0.009992109966941993
New Minimum: -550.5526892333164 > -550.55432481155
END: th(0.31001984126984145)=-550.55432481155; dx=-0.008459042172859997 evalInputDelta=0.0016355782336177072
Fitness changed from -550.5526892333164 to -550.55432481155
Iteration 15 complete. Error: -550.55432481155 Total: 2.0647; Orientation: 0.2885; Line Search: 1.2042
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -550.55432481155 (> -Infinity) after 46.210s (< 5400.000s)
Returns
-550.55432481155