1. SegmentStyle
  2. Resolution 600.0
  3. Resolution 1200.0

SegmentStyle

Demonstrates application of style transfer to a masked region identified by user scribble

Code from SegmentStyle.scala:73 executed in 0.00 seconds (0.000 gc):

    () => {
      implicit val _ = log
      // First, basic configuration so we publish to our s3 site
      log.setArchiveHome(URI.create(s"s3://$s3bucket/${getClass.getSimpleName.stripSuffix("$")}/${log.getId}/"))
      log.onComplete(() => upload(log): Unit)
      // Fetch input images (user upload prompts) and display a rescaled copies
      val (foreground: Tensor, background: Tensor) = log.subreport("Partition_Input", (subreport: NotebookOutput) => {
        var image = ImageArtUtil.load(subreport, contentUrl, partitioningResolution)
        val Seq(foreground, background) = partition(image, 2)(subreport)
        subreport.p(subreport.jpg(foreground.toImage, "Selection mask"))
        (foreground, background)
      })
      val styleImage = log.eval(()=>{
        ImageArtUtil.load(log, styleUrl, 600)
      })
      val canvas = new AtomicReference[Tensor](null)
      val registration = registerWithIndexJPG(canvas.get())
      try {
        withMonitoredJpg(() => canvas.get().toImage) {
          paint(contentUrl, content => {
            smoothStyle(content = content,
              style = Tensor.fromRGB(styleImage),
              contentMask = MomentMatcher.toMask(foreground.addRef()))
          }, canvas, new VisualStyleContentNetwork(
            styleLayers = List(
              VGG16.VGG16_0b,
              VGG16.VGG16_1a,
              VGG16.VGG16_1b1,
              VGG16.VGG16_1b2,
              VGG16.VGG16_1c1,
              VGG16.VGG16_1c2,
              VGG16.VGG16_1c3
            ),
            styleModifiers = List(
              new GramMatrixEnhancer(),
              new MomentMatcher()
            ).map(_.withMask(foreground)),
            styleUrl = List(styleUrl),
            contentLayers = List(
              VGG16.VGG16_1b2
            ),
            contentModifiers = List(
              new ContentMatcher()
            ),
            magnification = magnification
          ),
            new BasicOptimizer {
              override val trainingMinutes: Int = 180
              override val trainingIterations: Int = 30
              override val maxRate = 1e9
            }, new GeometricSequence {
              override val min: Double = minResolution
              override val max: Double = maxResolution
              override val steps = SegmentStyle.this.steps
            }.toStream.map(_.round.toDouble))
        }
      } finally {
        registration.foreach(_.stop()(s3client, ec2client))
      }
      null
    }

Returns:

    <function0>

Subreport: Partition_Input

Code from SegmentStyle.scala:86 executed in 213.53 seconds (0.000 gc):

    ImageArtUtil.load(log, styleUrl, 600)

Logging:

    <a id="p-6"></a><iframe src="input_544ee660-f999-4d0b-ae0f-dd12546590f3.html" id="544ee660-f999-4d0b-ae0f-dd12546590f3" style="margin: 0px; resize: both; overflow: auto; width: 100%; height: 90px;" sandbox="allow-forms allow-modals allow-pointer-lock allow-popups allow-presentation allow-same-origin allow-scripts" allow="geolocation; microphone; camera; midi; vr; accelerometer; gyroscope; payment; ambient-light-sensor; encrypted-media" scrolling="auto" allowtransparency="true" allowpaymentrequest="true" allowfullscreen="true"></iframe>
    
    

Returns:

Result

Resolution 600.0

Subreport: Optimization

Resolution 1200.0

Subreport: Optimization