BasicOptimizer.scala:89 executed in 90.91 seconds (1.768 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3512200505965
Reset training subject: 3512747933837
Adding measurement 574f87ac to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-15.50273737839654;dx=-8.003663641071308E-4
New Minimum: -15.50273737839654 > -15.502765535003704
WOLFE (weak): th(2.154434690031884)=-15.502765535003704; dx=-7.989282819366689E-4 evalInputDelta=2.815660716315449E-5
New Minimum: -15.502765535003704 > -15.502794331480859
WOLFE (weak): th(4.308869380063768)=-15.502794331480859; dx=-8.008915287853523E-4 evalInputDelta=5.695308431796775E-5
New Minimum: -15.502794331480859 > -15.502893518937197
WOLFE (weak): th(12.926608140191302)=-15.502893518937197; dx=-8.010610926973286E-4 evalInputDelta=1.5614054065693495E-4
New Minimum: -15.502893518937197 > -15.503368330099446
WOLFE (weak): th(51.70643256076521)=-15.503368330099446; dx=-8.003088656377714E-4 evalInputDelta=6.309517029059464E-4
New Minimum: -15.503368330099446 > -15.506202816589262
WOLFE (weak): th(258.53216280382605)=-15.506202816589262; dx=-7.935598293985274E-4 evalInputDelta=0.0034654381927214928
New Minimum: -15.506202816589262 > -15.526399049239615
WOLFE (weak): th(1551.1929768229563)=-15.526399049239615; dx=-7.682532115605876E-4 evalInputDelta=0.023661670843074134
New Minimum: -15.526399049239615 > -15.813845038053909
END: th(10858.350837760694)=-15.813845038053909; dx=-5.86420294024184E-4 evalInputDelta=0.31110765965736853
Fitness changed from -15.50273737839654 to -15.813845038053909
Iteration 1 complete. Error: -15.813845038053909 Total: 6.3371; Orientation: 0.0163; Line Search: 4.7098
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement c98407d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-15.813845038053909;dx=-7.308425773874474E-4
New Minimum: -15.813845038053909 > -16.00749678629971
END: th(23393.607721408407)=-16.00749678629971; dx=-4.1744621491844646E-4 evalInputDelta=0.19365174824580222
Fitness changed from -15.813845038053909 to -16.00749678629971
Iteration 2 complete. Error: -16.00749678629971 Total: 1.7132; Orientation: 0.0162; Line Search: 1.1392
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7e0999f2 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-16.00749678629971;dx=-6.863854984493533E-4
New Minimum: -16.00749678629971 > -17.484695805044854
END: th(50400.00000000001)=-17.484695805044854; dx=-2.5282892303023983E-4 evalInputDelta=1.477199018745143
Fitness changed from -16.00749678629971 to -17.484695805044854
Iteration 3 complete. Error: -17.484695805044854 Total: 1.7710; Orientation: 0.0170; Line Search: 1.2076
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement f00cdfe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.895e+03, gradient 2.742e-02, dot -0.357; [bd736ab2-e922-46fd-83eb-dda7e9aceb5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.484695805044854, -16.00749678629971, -15.813845038053909, -15.50273737839654
LBFGS Accumulation History: 3 points
Removed measurement f00cdfe to history. Total: 3
th(0)=-17.484695805044854;dx=-7.474010140903129E-4
New Minimum: -17.484695805044854 > -20.30326653377591
END: th(108583.50837760697)=-20.30326653377591; dx=-9.456377416302325E-5 evalInputDelta=2.818570728731057
Fitness changed from -17.484695805044854 to -20.30326653377591
Iteration 4 complete. Error: -20.30326653377591 Total: 2.2453; Orientation: 0.5268; Line Search: 1.1677
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 984f44d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.731e+03, gradient 2.920e-02, dot -0.176; [bd736ab2-e922-46fd-83eb-dda7e9aceb5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.30326653377591, -16.00749678629971, -15.813845038053909, -15.50273737839654
LBFGS Accumulation History: 3 points
Removed measurement 984f44d to history. Total: 3
th(0)=-20.30326653377591;dx=-8.493417047464137E-4
New Minimum: -20.30326653377591 > -23.837591045440746
END: th(233936.07721408413)=-23.837591045440746; dx=-1.9125007949834165E-5 evalInputDelta=3.5343245116648347
Fitness changed from -20.30326653377591 to -23.837591045440746
Iteration 5 complete. Error: -23.837591045440746 Total: 2.1145; Orientation: 0.4215; Line Search: 1.1446
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6aa41ae3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.692e+03, gradient 2.471e-02, dot -0.133; [bd736ab2-e922-46fd-83eb-dda7e9aceb5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.837591045440746, -16.00749678629971, -15.813845038053909, -15.50273737839654
LBFGS Accumulation History: 3 points
Removed measurement 6aa41ae3 to history. Total: 3
th(0)=-23.837591045440746;dx=-6.086750573769774E-4
Armijo: th(504000.0000000002)=-22.533548605528086; dx=-5.257125970195864E-6 evalInputDelta=-1.30404243991266
Armijo: th(252000.0000000001)=-23.788843948905967; dx=-5.0026211966973804E-5 evalInputDelta=-0.04874709653477893
New Minimum: -23.837591045440746 > -24.26769945871552
END: th(84000.00000000003)=-24.26769945871552; dx=-2.0649563256159813E-4 evalInputDelta=0.4301084132747732
Fitness changed from -23.837591045440746 to -24.26769945871552
Iteration 6 complete. Error: -24.26769945871552 Total: 3.2905; Orientation: 0.4166; Line Search: 2.3277
Adding measurement 19dedf45 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.160e+03, gradient 2.360e-02, dot -0.134; [bd736ab2-e922-46fd-83eb-dda7e9aceb5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.26769945871552, -16.00749678629971, -15.813845038053909, -15.50273737839654
LBFGS Accumulation History: 3 points
Removed measurement 19dedf45 to history. Total: 3
th(0)=-24.26769945871552;dx=-5.551555913182338E-4
New Minimum: -24.26769945871552 > -24.45216466253768
END: th(180972.5139626783)=-24.45216466253768; dx=-7.40474674385428E-5 evalInputDelta=0.18446520382216036
Fitness changed from -24.26769945871552 to -24.45216466253768
Iteration 7 complete. Error: -24.45216466253768 Total: 2.0814; Orientation: 0.4162; Line Search: 1.1439
Adding measurement 7bedf0a1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.459e+03, gradient 2.456e-02, dot -0.115; [bd736ab2-e922-46fd-83eb-dda7e9aceb5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.45216466253768, -16.00749678629971, -15.813845038053909, -15.50273737839654
LBFGS Accumulation History: 3 points
Removed measurement 7bedf0a1 to history. Total: 3
th(0)=-24.45216466253768;dx=-6.001255520274749E-4
New Minimum: -24.45216466253768 > -25.50896222834883
END: th(389893.4620234736)=-25.50896222834883; dx=-1.9137278018291904E-5 evalInputDelta=1.0567975658111521
Fitness changed from -24.45216466253768 to -25.50896222834883
Iteration 8 complete. Error: -25.50896222834883 Total: 2.7600; Orientation: 0.4171; Line Search: 1.8109
Adding measurement 4d56e1b7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.243e+03, gradient 1.992e-02, dot -0.082; [bd736ab2-e922-46fd-83eb-dda7e9aceb5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.50896222834883, -16.00749678629971, -15.813845038053909, -15.50273737839654
LBFGS Accumulation History: 3 points
Removed measurement 4d56e1b7 to history. Total: 3
th(0)=-25.50896222834883;dx=-3.9362389017001415E-4
Armijo: th(840000.0000000003)=-21.368422128600105; dx=-6.7866356438077604E-6 evalInputDelta=-4.140540099748726
Armijo: th(420000.0000000002)=-23.78332355275572; dx=-3.1839239863832685E-5 evalInputDelta=-1.7256386755931103
Armijo: th(140000.00000000006)=-25.346734186252245; dx=-1.503566012913379E-4 evalInputDelta=-0.16222804209658648
New Minimum: -25.50896222834883 > -25.75830119418417
END: th(35000.000000000015)=-25.75830119418417; dx=-3.15015749321195E-4 evalInputDelta=0.24933896583533866
Fitness chan
...skipping 9105 bytes...
aceb5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.414277836917282, -16.00749678629971, -15.813845038053909, -15.50273737839654
LBFGS Accumulation History: 3 points
Removed measurement 193d6ed7 to history. Total: 3
th(0)=-29.414277836917282;dx=-2.76493252502692E-4
Armijo: th(752109.3017428122)=-25.122899285264737; dx=-1.6195338197462497E-5 evalInputDelta=-4.291378551652546
Armijo: th(376054.6508714061)=-26.923753454339302; dx=-5.435811938693022E-5 evalInputDelta=-2.4905243825779806
Armijo: th(125351.55029046869)=-28.831558883346673; dx=-1.4766262235488644E-4 evalInputDelta=-0.582718953570609
New Minimum: -29.414277836917282 > -29.42633210951516
END: th(31337.887572617172)=-29.42633210951516; dx=-2.2623294999394553E-4 evalInputDelta=0.01205427259787939
Fitness changed from -29.414277836917282 to -29.42633210951516
Iteration 20 complete. Error: -29.42633210951516 Total: 4.4243; Orientation: 0.4149; Line Search: 3.5073
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Adding measurement 3dd1a88a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.371e+04, gradient 1.658e-02, dot -0.148; [bd736ab2-e922-46fd-83eb-dda7e9aceb5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.42633210951516, -16.00749678629971, -15.813845038053909, -15.50273737839654
LBFGS Accumulation History: 3 points
Removed measurement 3dd1a88a to history. Total: 3
th(0)=-29.42633210951516;dx=-2.7369937881593844E-4
Armijo: th(67515.4320987655)=-29.0787361116477; dx=-1.9242506281711032E-4 evalInputDelta=-0.34759599786746165
Armijo: th(33757.71604938275)=-29.331081032628106; dx=-2.2114752820205673E-4 evalInputDelta=-0.09525107688705603
Armijo: th(11252.572016460916)=-29.406677478133396; dx=-2.4666048243851654E-4 evalInputDelta=-0.019654631381765597
Armijo: th(2813.143004115229)=-29.420151151981027; dx=-2.568261321715819E-4 evalInputDelta=-0.006180957534134279
Armijo: th(562.6286008230458)=-29.4250766022371; dx=-2.583491164386599E-4 evalInputDelta=-0.0012555072780600085
Armijo: th(93.77143347050763)=-29.42610151015349; dx=-2.588237584337329E-4 evalInputDelta=-2.3059936167157957E-4
Armijo: th(13.395919067215376)=-29.42629704774041; dx=-2.585492248402742E-4 evalInputDelta=-3.506177475287586E-5
Armijo: th(1.674489883401922)=-29.426324018340082; dx=-2.5803715117186016E-4 evalInputDelta=-8.091175079272261E-6
Armijo: th(0.18605443148910245)=-29.426326715398687; dx=-2.587140556904242E-4 evalInputDelta=-5.394116474377597E-6
Armijo: th(0.018605443148910245)=-29.426326715398687; dx=-2.5874553721816546E-4 evalInputDelta=-5.394116474377597E-6
Armijo: th(0.001691403922628204)=-29.426330760986136; dx=-2.58780487340922E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(1.4095032688568368E-4)=-29.42633210951516; dx=-2.5908410845524145E-4 evalInputDelta=0.0
Armijo: th(1.0842332837360284E-5)=-29.42633210951516; dx=-2.584369770668193E-4 evalInputDelta=0.0
WOLFE (weak): th(7.744523455257346E-7)=-29.42633210951516; dx=-2.59088279569191E-4 evalInputDelta=0.0
Armijo: th(5.808392591443009E-6)=-29.426330760986136; dx=-2.5909146140129936E-4 evalInputDelta=-1.3485290253356652E-6
WOLFE (weak): th(3.291422468484372E-6)=-29.42633210951516; dx=-2.59184333531342E-4 evalInputDelta=0.0
WOLFE (weak): th(4.5499075299636905E-6)=-29.42633210951516; dx=-2.592228339166134E-4 evalInputDelta=0.0
WOLFE (weak): th(5.17915006070335E-6)=-29.42633210951516; dx=-2.587012315492623E-4 evalInputDelta=0.0
Armijo: th(5.49377132607318E-6)=-29.426330760986136; dx=-2.589159588267893E-4 evalInputDelta=-1.3485290253356652E-6
WOLFE (weak): th(5.336460693388265E-6)=-29.42633210951516; dx=-2.5866785508197167E-4 evalInputDelta=0.0
Armijo: th(5.415116009730722E-6)=-29.426330760986136; dx=-2.589127289859677E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.375788351559493E-6)=-29.426330760986136; dx=-2.589715741645323E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.3561245224738794E-6)=-29.426330760986136; dx=-2.588437455300359E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.346292607931072E-6)=-29.426330760986136; dx=-2.5886830580229764E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.3413766506596684E-6)=-29.426330760986136; dx=-2.587785257365079E-4 evalInputDelta=-1.3485290253356652E-6
WOLFE (weak): th(5.338918672023967E-6)=-29.42633210951516; dx=-2.5881638816330454E-4 evalInputDelta=0.0
Armijo: th(5.3401476613418175E-6)=-29.426330760986136; dx=-2.5904131173408126E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.339533166682892E-6)=-29.426330760986136; dx=-2.5927975362308484E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.33922591935343E-6)=-29.426330760986136; dx=-2.5902972491587616E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.339072295688698E-6)=-29.426330760986136; dx=-2.5870638289913697E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.338995483856332E-6)=-29.426330760986136; dx=-2.5858867232726203E-4 evalInputDelta=-1.3485290253356652E-6
WOLFE (weak): th(5.33895707794015E-6)=-29.42633210951516; dx=-2.585064278241168E-4 evalInputDelta=0.0
mu ~= nu (5.33895707794015E-6): th(0.0)=-29.42633210951516
Fitness changed from -29.42633210951516 to -29.42633210951516
Static Iteration Total: 20.9161; Orientation: 0.4237; Line Search: 19.9438
Iteration 21 failed. Error: -29.42633210951516
Previous Error: 0.0 -> -29.42633210951516
Retrying iteration 21
Reset training subject: 3588542572712
Adding measurement 477bd98d to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-29.42633210951516;dx=-2.733432981762737E-4
Armijo: th(1.1502475708824582E-5)=-29.426330760986136; dx=-2.588317147974781E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.751237854412291E-6)=-29.426330760986136; dx=-2.5855844507852816E-4 evalInputDelta=-1.3485290253356652E-6
WOLFE (weak): th(1.917079284804097E-6)=-29.42633210951516; dx=-2.5854700916729906E-4 evalInputDelta=0.0
WOLFE (weak): th(3.834158569608194E-6)=-29.42633210951516; dx=-2.5820225015284545E-4 evalInputDelta=0.0
WOLFE (weak): th(4.792698212010243E-6)=-29.42633210951516; dx=-2.5819655935462553E-4 evalInputDelta=0.0
WOLFE (weak): th(5.271968033211267E-6)=-29.42633210951516; dx=-2.587819744819512E-4 evalInputDelta=0.0
Armijo: th(5.5116029438117795E-6)=-29.426330760986136; dx=-2.589103737845261E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.391785488511523E-6)=-29.426330760986136; dx=-2.587001367461659E-4 evalInputDelta=-1.3485290253356652E-6
WOLFE (weak): th(5.331876760861395E-6)=-29.42633210951516; dx=-2.587050686033315E-4 evalInputDelta=0.0
Armijo: th(5.361831124686459E-6)=-29.426330760986136; dx=-2.5871149373912306E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.346853942773928E-6)=-29.426330760986136; dx=-2.585576636368375E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.3393653518176615E-6)=-29.426330760986136; dx=-2.5874733758900827E-4 evalInputDelta=-1.3485290253356652E-6
WOLFE (weak): th(5.3356210563395284E-6)=-29.42633210951516; dx=-2.5853867961465096E-4 evalInputDelta=0.0
WOLFE (weak): th(5.337493204078595E-6)=-29.42633210951516; dx=-2.586933452396384E-4 evalInputDelta=0.0
WOLFE (weak): th(5.338429277948128E-6)=-29.42633210951516; dx=-2.586873081057042E-4 evalInputDelta=0.0
WOLFE (weak): th(5.338897314882895E-6)=-29.42633210951516; dx=-2.58949050944891E-4 evalInputDelta=0.0
Armijo: th(5.339131333350278E-6)=-29.426330760986136; dx=-2.5888731543304185E-4 evalInputDelta=-1.3485290253356652E-6
Armijo: th(5.339014324116586E-6)=-29.426330760986136; dx=-2.586797299822252E-4 evalInputDelta=-1.3485290253356652E-6
WOLFE (weak): th(5.338955819499741E-6)=-29.42633210951516; dx=-2.586766595993054E-4 evalInputDelta=0.0
WOLFE (weak): th(5.338985071808163E-6)=-29.42633210951516; dx=-2.5855579015213013E-4 evalInputDelta=0.0
mu ~= nu (5.338985071808163E-6): th(0.0)=-29.42633210951516
Fitness changed from -29.42633210951516 to -29.42633210951516
Static Iteration Total: 14.5638; Orientation: 0.0161; Line Search: 13.4867
Iteration 22 failed. Error: -29.42633210951516
Previous Error: 0.0 -> -29.42633210951516
Optimization terminated 22
Final threshold in iteration 22: -29.42633210951516 (> -Infinity) after 90.906s (< 3600.000s)
Returns
-29.42633210951516