BasicOptimizer.scala:89 executed in 8.25 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 167928725244300
Reset training subject: 167928856362200
Adding measurement 3146cc16 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.0953277508201892;dx=-9.915711791739828E-7
New Minimum: -0.0953277508201892 > -0.0953299504593329
WOLFE (weak): th(2.154434690031884)=-0.0953299504593329; dx=-9.89775916306587E-7 evalInputDelta=2.1996391437045792E-6
New Minimum: -0.0953299504593329 > -0.09533220863170823
WOLFE (weak): th(4.308869380063768)=-0.09533220863170823; dx=-9.970431180584587E-7 evalInputDelta=4.457811519031907E-6
New Minimum: -0.09533220863170823 > -0.09534108102690524
WOLFE (weak): th(12.926608140191302)=-0.09534108102690524; dx=-9.91579206149107E-7 evalInputDelta=1.3330206716039017E-5
New Minimum: -0.09534108102690524 > -0.09538105514599818
WOLFE (weak): th(51.70643256076521)=-0.09538105514599818; dx=-9.926108964649717E-7 evalInputDelta=5.3304325808981035E-5
New Minimum: -0.09538105514599818 > -0.09559507994824816
WOLFE (weak): th(258.53216280382605)=-0.09559507994824816; dx=-9.91896061486396E-7 evalInputDelta=2.673291280589629E-4
New Minimum: -0.09559507994824816 > -0.09692432233044472
WOLFE (weak): th(1551.1929768229563)=-0.09692432233044472; dx=-9.932048656616183E-7 evalInputDelta=0.0015965715102555234
New Minimum: -0.09692432233044472 > -0.10672332753482949
WOLFE (weak): th(10858.350837760694)=-0.10672332753482949; dx=-9.600480564410386E-7 evalInputDelta=0.011395576714640296
New Minimum: -0.10672332753482949 > -0.1726607186205185
END: th(86866.80670208555)=-0.1726607186205185; dx=-4.4202149249602175E-7 evalInputDelta=0.07733296780032929
Fitness changed from -0.0953277508201892 to -0.1726607186205185
Iteration 1 complete. Error: -0.1726607186205185 Total: 1.4474; Orientation: 0.0013; Line Search: 1.0776
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 48224143 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.1726607186205185;dx=-1.0194864607526533E-6
New Minimum: -0.1726607186205185 > -0.26518422934040475
WOLF (strong): th(187148.86177126726)=-0.26518422934040475; dx=2.0154699390820814E-8 evalInputDelta=0.09252351071988626
END: th(93574.43088563363)=-0.26179348367199656; dx=-3.5711722063345024E-7 evalInputDelta=0.08913276505147807
Fitness changed from -0.1726607186205185 to -0.26518422934040475
Iteration 2 complete. Error: -0.26518422934040475 Total: 0.4672; Orientation: 0.0014; Line Search: 0.3471
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4eb08135 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.26518422934040475;dx=-1.7815849740533526E-6
New Minimum: -0.26518422934040475 > -0.3119861858503622
WOLF (strong): th(201600.00000000003)=-0.3119861858503622; dx=3.436013168650542E-7 evalInputDelta=0.04680195650995744
New Minimum: -0.3119861858503622 > -0.38713317159272614
END: th(100800.00000000001)=-0.38713317159272614; dx=-7.640444830298883E-8 evalInputDelta=0.12194894225232139
Fitness changed from -0.26518422934040475 to -0.38713317159272614
Iteration 3 complete. Error: -0.38713317159272614 Total: 0.4750; Orientation: 0.0016; Line Search: 0.3504
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1e8d29fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.162e+02, gradient 1.146e-03, dot -0.350; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.38713317159272614, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 1e8d29fe to history. Total: 3
th(0)=-0.38713317159272614;dx=-1.2911308174825768E-6
New Minimum: -0.38713317159272614 > -0.4326469695566475
WOLF (strong): th(217167.01675521393)=-0.4326469695566475; dx=1.279174466924675E-7 evalInputDelta=0.045513797963921354
New Minimum: -0.4326469695566475 > -0.46642591758569746
END: th(108583.50837760697)=-0.46642591758569746; dx=-1.7543938176979997E-7 evalInputDelta=0.07929274599297131
Fitness changed from -0.38713317159272614 to -0.46642591758569746
Iteration 4 complete. Error: -0.46642591758569746 Total: 0.4844; Orientation: 0.0277; Line Search: 0.3395
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 72fb9b5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.681e+02, gradient 1.080e-03, dot -0.381; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.46642591758569746, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 72fb9b5 to history. Total: 3
th(0)=-0.46642591758569746;dx=-1.139587121816142E-6
New Minimum: -0.46642591758569746 > -0.49425766543935656
WOLF (strong): th(233936.07721408413)=-0.49425766543935656; dx=1.2009706682875503E-7 evalInputDelta=0.027831747853659106
New Minimum: -0.49425766543935656 > -0.5252149579865614
END: th(116968.03860704206)=-0.5252149579865614; dx=-1.1696540313401131E-7 evalInputDelta=0.05878904040086391
Fitness changed from -0.46642591758569746 to -0.5252149579865614
Iteration 5 complete. Error: -0.5252149579865614 Total: 0.4942; Orientation: 0.0299; Line Search: 0.3473
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5b0e72f7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.667e+02, gradient 1.091e-03, dot -0.361; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.5252149579865614, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 5b0e72f7 to history. Total: 3
th(0)=-0.5252149579865614;dx=-1.1560128506873861E-6
New Minimum: -0.5252149579865614 > -0.5299317500891403
WOLF (strong): th(252000.0000000001)=-0.5299317500891403; dx=1.908266048499244E-7 evalInputDelta=0.004716792102578915
New Minimum: -0.5299317500891403 > -0.5829049920488589
WOLF (strong): th(126000.00000000004)=-0.5829049920488589; dx=4.148961900659312E-9 evalInputDelta=0.05769003406229756
END: th(42000.000000000015)=-0.5742581224828074; dx=-5.890830616416504E-7 evalInputDelta=0.04904316449624602
Fitness changed from -0.5252149579865614 to -0.5829049920488589
Iteration 6 complete. Error: -0.5829049920488589 Total: 0.6081; Orientation: 0.0330; Line Search: 0.4541
Adding measurement 3804efbd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.043e+03, gradient 1.265e-03, dot -0.319; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.5829049920488589, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 3804efbd to history. Total: 3
th(0)=-0.5829049920488589;dx=-1.5708380073198464E-6
New Minimum: -0.5829049920488589 > -0.6177341322072702
WOLF (strong): th(90486.25698133915)=-0.6177341322072702; dx=2.160922201625206E-7 evalInputDelta=0.03482914015841132
New Minimum: -0.6177341322072702 > -0.6366506994084146
END: th(45243.12849066957)=-0.6366506994084146; dx=-2.0484316370107174E-7 evalInputDelta=0.05374570735955564
Fitness changed from -0.5829049920488589 to -0.6366506994084146
Iteration 7 complete. Error: -0.6366506994084146 Total: 0.4991; Orientation: 0.0285; Line Search: 0.3475
Adding measurement 401b4329 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.299e+03, gradient 1.020e-03, dot -0.383; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.6366506994084146, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 401b4329 to history. Total: 3
th(0)=-0.6366506994084146;dx=-1.0064485923486232E-6
New Minimum: -0.6366506994084146 > -0.639483253704028
WOLF (strong): th(97473.3655058684)=-0.639483253704028; dx=1.7784425806199863E-7 evalInputDelta=0.0028325542956134653
New Minimum: -0.639483253704028 > -0.6625642920104834
END: th(48736.6827529342)=-0.6625642920104834; dx=-1.6784056909231457E-7 evalInputDelta=0.025913592602068825
Fitness changed from -0.6366506994084146 to -0.6625642920104834
Iteration 8 complete. Error: -0.6625642920104834 Total: 0.4657; Orientation: 0.0260; Line Search: 0.3306
Adding measurement 502f6931 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.166e+03, gradient 9.197e-04, dot -0.406; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.6625642920104834, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 502f6931 to history. Total: 3
th(0)=-0.6625642920104834;dx=-8.253388326307246E-7
New Minimum: -0.6625642920104834 > -0.6902670608978385
WOLF (strong): th(105000.00000000004)=-0.6902670608978385; dx=3.679916253685042E-8 evalInputDelta=0.027702768887355123
New Minimum: -0.6902670608978385 > -0.6947359291377303
END: th(52500.00000000002)=-0.6947359291377303; dx=-2.3885664216666443E-7 evalInputDelta=0.03217163712724691
Fitness changed from -0.6625642920104834 to -0.6947359291377303
Iteration 9 complete. Error: -0.6947359291377303 Total: 0.4660; Orientation: 0.0265; Line Search: 0.3308
Adding measurement 49adcfec to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.323e+03, gradient 8.343e-04, dot -0.436; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.6947359291377303, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 49adcfec to history. Total: 3
th(0)=-0.6947359291377303;dx=-6.778465967945213E-7
New Minimum: -0.6947359291377303 > -0.7041422946404805
WOLF (strong): th(113107.82122667394)=-0.7041422946404805; dx=2.75286336596902E-8 evalInputDelta=0.009406365502750202
New Minimum: -0.7041422946404805 > -0.7166050378731539
END: th(56553.91061333697)=-0.7166050378731539; dx=-2.2022190637516251E-7 evalInputDelta=0.02186910873542358
Fitness changed from -0.6947359291377303 to -0.7166050378731539
Iteration 10 complete. Error: -0.7166050378731539 Total: 0.4647; Orientation: 0.0261; Line Search: 0.3292
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 49e54dab to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.324e+03, gradient 8.051e-04, dot -0.446; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.7166050378731539, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 49e54dab to history. Total: 3
th(0)=-0.7166050378731539;dx=-6.311969093680947E-7
New Minimum: -0.7166050378731539 > -0.7301618577564618
WOLF (strong): th(121841.70688233551)=-0.7301618577564618; dx=1.1984535459507814E-7 evalInputDelta=0.013556819883307925
New Minimum: -0.7301618577564618 > -0.7444161910064611
END: th(60920.853441167754)=-0.7444161910064611; dx=-1.6056741358078333E-7 evalInputDelta=0.027811153133307176
Fitness changed from -0.7166050378731539 to -0.7444161910064611
Iteration 11 complete. Error: -0.7444161910064611 Total: 0.4730; Orientation: 0.0265; Line Search: 0.3295
Adding measurement 6461fc60 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.688e+03, gradient 8.179e-04, dot -0.455; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.7444161910064611, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 6461fc60 to history. Total: 3
th(0)=-0.7444161910064611;dx=-6.433476218800825E-7
Armijo: th(131250.00000000006)=-0.7435840079176702; dx=4.5912675621175167E-8 evalInputDelta=-8.321830887908499E-4
New Minimum: -0.7444161910064611 > -0.7602154300853756
END: th(65625.00000000003)=-0.7602154300853756; dx=-8.140854947568068E-8 evalInputDelta=0.015799239078914562
Fitness changed from -0.7444161910064611 to -0.7602154300853756
Iteration 12 complete. Error: -0.7602154300853756 Total: 0.4827; Orientation: 0.0290; Line Search: 0.3402
Adding measurement 43d5edab to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.646e+03, gradient 8.646e-04, dot -0.425; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.7602154300853756, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 43d5edab to history. Total: 3
th(0)=-0.7602154300853756;dx=-7.282655334008741E-7
New Minimum: -0.7602154300853756 > -0.7674729394675016
WOLF (strong): th(141384.77653334243)=-0.7674729394675016; dx=1.6170855732300086E-7 evalInputDelta=0.007257509382125993
New Minimum: -0.7674729394675016 > -0.7918786387111952
END: th(70692.38826667122)=-0.7918786387111952; dx=-2.2836809858960875E-8 evalInputDelta=0.03166320862581962
Fitness changed from -0.7602154300853756 to -0.7918786387111952
Iteration 13 complete. Error: -0.7918786387111952 Total: 0.4714; Orientation: 0.0280; Line Search: 0.3322
Adding measurement 40842a93 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.182e+03, gradient 8.830e-04, dot -0.422; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.7918786387111952, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 40842a93 to history. Total: 3
th(0)=-0.7918786387111952;dx=-7.416902981078063E-7
Armijo: th(152302.13360291938)=-0.7724213004666068; dx=2.604163951276371E-7 evalInputDelta=-0.019457338244588396
New Minimum: -0.7918786387111952 > -0.8122242075089144
WOLF (strong): th(76151.06680145969)=-0.8122242075089144; dx=3.184408921956555E-8 evalInputDelta=0.020345568797719205
END: th(25383.688933819896)=-0.8089843461874563; dx=-4.413366499767644E-7 evalInputDelta=0.017105707476261056
Fitness changed from -0.7918786387111952 to -0.8122242075089144
Iteration 14 complete. Error: -0.8122242075089144 Total: 0.5785; Orientation: 0.0280; Line Search: 0.4448
Adding measurement 3d9ea4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.253e+03, gradient 9.053e-04, dot -0.412; [690362c6-96a0-4fba-b0b5-b80057213fcc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.8122242075089144, -0.26518422934040475, -0.1726607186205185, -0.0953277508201892
LBFGS Accumulation History: 3 points
Removed measurement 3d9ea4 to history. Total: 3
th(0)=-0.8122242075089144;dx=-7.899983177186721E-7
New Minimum: -0.8122242075089144 > -0.8453597802204487
END: th(54687.50000000003)=-0.8453597802204487; dx=-1.718532811925691E-7 evalInputDelta=0.03313557271153422
Fitness changed from -0.8122242075089144 to -0.8453597802204487
Iteration 15 complete. Error: -0.8453597802204487 Total: 0.3664; Orientation: 0.0262; Line Search: 0.2300
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -0.8453597802204487 (> -Infinity) after 8.254s (< 5400.000s)
Returns
-0.8453597802204487