Lab book Fluotracify

1 Technical Notes

1.1 README

1.1.1 General:

  • This file corresponds to my lab book for my doctoral thesis tackling artifact correction in Fluorescence Correlation Spectroscopy (FCS) measurements using Deep Neural Networks. It also contains notes taken during the process of setting up this workflow for reproducible research.
  • This file contains explanations of how things are organized, of the workflow for doing experiments, changes made to the code, and the observed behavior in the "* Data" section.
  • The branching model used is described in this paper. Therefore: if you are interested in the "* Data" section, you have to git clone the data branch of the repository. The main branch is clean from any results, it contains only source code and the analysis.
  • This project is my take on Open-notebook science. The idea was postulated in a blog post in 2006:

    … there is a URL to a laboratory notebook that is freely available and indexed on common search engines. It does not necessarily have to look like a paper notebook but it is essential that all of the information available to the researchers to make their conclusions is equally available to the rest of the world —Jean-Claude Bradley

  • Proposal on how to deal with truly private data (e.g. notes from a confidential meeting with a colleague), which might otherwise be noted in a normal Lab notebook: do not include them here. Only notes relevant to the current project should be taken

1.1.2 Code block languages used in this document

# This is a sh block for shell / bash scripting. In the context of this file,
# these blocks are mainly used for operations on my local computer.
# In the LabBook.html rendering of this document, these blocks will have a
# light green colour (#F0FBE9)

# This block can open and access tmux sessions, used for shell scripting on
# remote computing clusters.
# In the LabBook.html rendering of this document, these blocks will have a
# distinct light green colour (#E1EED8)

# This is a python block. In the context of this file, it is seldomly used
# (only for examplary scripts.)
# In the LabBook.html rendering of this document, these blocks will have a
# light blue colour (#E6EDF4)

# This is a jupyter-python block. The code is sent to a jupyter kernel running
# on a remote high performance computing cluster. Most of my jupyter code is
# executed this way.
# In the LabBook.html rendering of this document, these blocks will have a
# light orange colour (#FAEAE1)

;; This is a emacs-lisp block, the language used to customize Emacs, which is
;; sometimes necessary, since the reproducible workflow of this LabBook is
;; tightly integrated with Emacs and org-mode.
;; In the LabBook.html rendering of this document, these blocks will have a
;; light violet colour (#F7ECFB)

This is a literal example block. It can be used very flexibly - in the context
of this document the output of most code blocks is displayed this way.
In the LabBook.html rendering of this document, these blocks will have a light
yellow colour (#FBFBBF)

This is a literal example block enclosed in a details block. This is useful to
make the page more readable by collapsing large amounts of output.
In the Labbook.html rendering of this document, the details block will have a
light grey colour (#f0f0f0) and a pink color when hovering above it.

1.1.3 Experiments workflow:

  1. Create a new branch from main
  2. Print out the git log from the latest commit and the metadata
  3. Call the analysis scripts, follow the principles outlined in Organization of code
  4. All machine learning runs are saved in data/mlruns, all other data in data/#experiment-name
  5. Add a ** exp-<date>-<name>" section to this file under Data
  6. Commit/push the results of this separate branch
  7. Merge this new branch with the remote data branch

1.1.4 Example for experimental setup procedure

1.1.4.1 Setting starting a jupyter kernel from a remote jupyter session using emacs-jupyter in org babel

1.1.5 tools used (notes)

1.1.5.1 Emacs magit
1.1.5.2 jupyter

1.2 Template for data entry and setup notes:

1.2.1 exp-#date-#title

1.2.1.1 git:
git log -1
1.2.1.2 System Metadata:
import os
import pprint

ramlist = os.popen('free -th').readlines()[-1].split()[1:]

print('No of CPUs in system:', os.cpu_count())
print('No of CPUs the current process can use:',
      len(os.sched_getaffinity(0)))
print('load average:', os.getloadavg())
print('os.uname(): ', os.uname())
print('PID of process:', os.getpid())
print('RAM total: {}, RAM used: {}, RAM free: {}'.format(
    ramlist[0], ramlist[1], ramlist[2]))

!echo the current directory: $PWD
!echo My disk usage:
!df -h
if _long:
    %conda list
    pprint.pprint(dict(os.environ), sort_dicts=False)
1.2.1.3 Tmux setup and scripts
rm ~/.tmux-local-socket-remote-machine
REMOTE_SOCKET=$(ssh ara 'tmux ls -F "#{socket_path}"' | head -1)
echo $REMOTE_SOCKET
ssh ara -tfN \
    -L ~/.tmux-local-socket-remote-machine:$REMOTE_SOCKET
rm: cannot remove 'home/lex.tmux-local-socket-remote-machine': No such file or directory
ye53nis@ara-login01.rz.uni-jena.de's password:              
/tmp/tmux-67339/default                
> ye53nis@ara-login01.rz.uni-jena.de's password:            
1.2.1.4 SSH tunneling

Different applications can be run on the remote compute node. If I want to access them at the local machine, and open them with the browser, I use this tunneling script. 

ssh -t -t ara -L $port:localhost:$port ssh $node -L $port:Localhost:$port

Apps I use that way:

  • Jupyter lab for running Python 3-Kernels
  • TensorBoard
  • Mlflow ui
1.2.1.5 jupyter scripts

Starting a jupyter instance on a server where the necessary libraries are installed is easy using this script: 

conda activate tf
export PORT=9999
export XDG_RUNTIME_DIR=''
export XDG_RUNTIME_DIR=""
jupyter lab --no-browser --port=$PORT

On the compute node of the HPC, the users' environment is managed through module files using the system Lmod. The export XDG_RUNTIME_DIR statements are needed because of a jupyter bug which did not let it start. Right now, ob-tmux does not support a :var header like normal org-babel does. So the $port variable has to be set here in the template.

Now this port has to be tunnelled on our local computer (See SSH tunneling). While the tmux session above keeps running, no matter if Emacs is running or not, this following ssh tunnel needs to be active locally to connect to the notebook. If you close Emacs, it would need to be reestablished

1.2.2 Setup notes

1.2.2.1 Setting up a tmux connection from using ob-tmux in org-babel
  • prerequisite: tmux versions need to be the same locally and on the server. Let's verify that now.
    • the local tmux version: 
      tmux -V

      tmux 3.0a
    • the remote tmux version: 
      ssh ara tmux -V
      ye53nis@ara-login01.rz.uni-jena.de's password:
      tmux 3.0a
  • as is described in the ob-tmux readme, the following code snippet creates a socket on the remote machine and forwards this socket to the local machine (note that socket_path was introduced in tmux version 2.2) 
    REMOTE_SOCKET=$(ssh ara 'tmux ls -F "#{socket_path}"' | head -1)
echo $REMOTE_SOCKET
ssh ara -tfN \
    -L ~/.tmux-local-socket-remote-machine:$REMOTE_SOCKET
    ye53nis@ara-login01.rz.uni-jena.de's password:  
    /tmp/tmux-67339/default    
    > ye53nis@ara-login01.rz.uni-jena.de's password:
  • now a new tmux session with name ob-NAME is created when using a code block which looks like this: #+BEGIN_SRC tmux :socket ~/.tmux-local-socket-remote-machine :session NAME
  • Commands can be sent now to the remote tmux session, BUT note that the output is not printed yet
  • there is a workaround for getting output back to our LabBook.org: A script which allows to print the output from the tmux session in an #+begin_example-Block below the tmux block by pressing C-c C-o or C-c C-v C-o when the pointer is inside the tmux block.
1.2.2.2 emacs-jupyter Setup

Emacs-jupyter aims to be an API for a lot of functionalities of the jupyter project. The documentation can be found on GitHub.

  1. For the whole document: connect to a running jupyter instance
    1. M-x jupyter-server-list-kernels
      1. set server URL, e.g. http://localhost:8889
      2. set websocket URL, e.g. http://localhost:8889
    2. two possibilities
      1. kernel already exists \(\to\) list of kernels and kernel-ID is displayed
      2. kernel does not exist \(\to\) prompt asks if you want to start one \(\to\) yes \(\to\) type kernel you want to start, e.g. Python 3
  2. In the subtree where you want to use jupyter-python blocks with org babel
    1. set the :header-args:jupyter-python :session /jpy:localhost#kernel:8889-ID
    2. customize the output folder using the following org-mode variable: 
      (setq org-babel-jupyter-resource-directory "./data/exp-test/plots")

      ./data/exp-test/plots
  3. For each individual block, the following customizations might be useful
    1. jupyter kernels can return multiple kinds of rich output (images, html, …) or scalar data (plain text, numbers, lists, …). To force a plain output, use :results scalar. To show the output in the minibuffer only, use :results silent
    2. to change the priority of different rich outputs, use :display header argument, e.g. :display text/plain text/html prioritizes plain text over html. All supported mimetypes in default order:
      1. text/org
      2. image/svg+xml, image/jpeg, image/png
      3. text/html
      4. text/markdown
      5. text/latex
      6. text/plain
    3. We can set jupyter to output pandas DataFrames as org tables automatically using the source block header argument :pandoc t
    4. useful keybindings
      • M-i to open the documentation for wherever your pointer is (like pressing Shift-TAB in Jupyter notebooks)
      • C-c C-i to interrupt the kernel, C-c C-r to restart the kernel

1.2.3 Notes on archiving

1.2.3.1 Exporting the LabBook.org to html in a twbs style
  • I am partial to the twitter bootstrap theme of html, since I like it's simple design, but clear structure with a nice table of contents at the side → the following org mode extension supports a seemless export to twitter bootstrap html: https://github.com/marsmining/ox-twbs
  • when installed, the export can be triggered via the command (org-twbs-export-as-html) or via the keyboard shortcut for export C-c C-e followed by w for Twitter bootstrap and h for saving the .html
  • Things to configure:
    • in general, there are multiple export options: https://orgmode.org/manual/Export-Settings.html
    • E.g. I set 2 #+OPTIONS keywords at the begin of the file: toc:4 and H:4 which make sure that in my export my sidebar table of contents will show numbered headings till a depth of 4.
    • I configured my code blocks so that they will not be evaluated when exporting (I would recommend this especially if you only export for archiving) and that both the code block and the output will be exported with the keyword: #+PROPERTY: header-args :eval never-export :exports both
    • To discriminate between code blocks for different languages I gave each of them a distinct colour using #+HTML_HEAD_EXTRA: <style... (see above)
    • I had to configure a style for table, so that the
      • display: block; overflow-x: auto; gets the table to be restricted to the width of the text and if it is larger, activates scrolling
      • white-space: nowrap; makes it that there is no wrap in a column, so it might be broader, but better readable if you have scrolling anyway
  • Things to do before exporting / Troubleshooting while exporting:
    • when using a dark theme for you emacs, the export of the code blocks might show some ugly dark backgrounds from the theme. If this becomes an issue, change to a light theme for the export with M-x (load-theme) and choose solarized-light
    • only in the data branch you set the git tags after merging. If you want to show them here, execute the corresponding function in Git TAGs
    • make sure your file links work properly! I recommend referencing your files relatively (e.g. [ [ f ile:./data/exp-XXXXXX-test/test.png]] without spaces). Otherwise there will be errors in your Messages buffer
    • There might be errors with your code blocks
      • e.g. the export function expects you to assign a default variable to your functions
      • if you call a function via the #+CALL mechanism, it wants you to include two parentheses for the function, e.g. #+CALL: test()
    • check indentation of code blocks inside lists
    • add a details block around large output cells. This makes them expandable. I added some #+HTML_HEAD_EXTRA: <style... inspired by alhassy. That's how the details block looks like: 
      #+begin_details

#+end_details
    • If you reference a parameter with an underscore in the name, use the org markdown tricks to style them like code (== or ~~), otherwise the part after the underscore will be rendered like a subscript: under_score vs underscore
  • Things to do after exporting:
    • In my workflow, the exported LabBook.html with the overview of all experiments is in the data folder. If you move the file, you will have to fix the file links for the new location, e.g. via "Find and replace" M-%:
      • if you move the org file → in the org file find [[file:./data/ and replace with [[file:./ → then export with C-c C-e w h
      • if you export first with C-c C-e w h and move the html file to data → in the html file find ./data and replace with .

1.3 Organization of git

1.3.1 remote/origin/main branch:

  • contains all the source code in folder src/ which is used for experiments.
  • contains the LabBook.org template
  • contains setup- and metadata files such as MLproject or conda.yaml
  • the log contains only lasting alterations on the folders and files mentioned above, which are e.g. used for conducting experiments or which introduce new features. Day-to-day changes in code

1.3.2 remote/origin/exp### branches:

  • if an experiment is done, the code and templates will be branched out from main in an #experiment-name branch, ### meaning some meaningful descriptor.
  • all data generated during the experiment (e.g. .csv files, plots, images, etc), is stored in a folder with the name data/#experiment-name, except machine learning-specific data and metadata from `mlflow` runs, which are saved under data/mlruns (this allows easily comparing machine learning runs with different experimental settings)
  • The LabBook.org file is essential
    • If possible, all code is executed from inside this file (meaning analysis scripts or calling the code from the scr/ directory).
    • All other steps taken during an experiment are noted down, as well as conclusions or my thought process while conducting the experiment
    • Provenance data, such as metadata about the environment the code was executed in, the command line output of the code, and some plots

1.3.3 remote/origin/develop branch:

  • this is the branch I use for day to day work on features and exploration. All of my current activity can be followed here.

1.3.4 remote/origin/data branch:

  • contains a full cronicle of the whole research process
  • all #experiment-name branches are merged here. Afterwards the original branch is deleted and on the data branch there is a Git tag which shows the merge commit to make accessing single experiments easy.
  • the develop branch is merged here as well.

1.3.5 Git TAGs

1.3.5.1 Stable versions:
1.3.5.2 All tags from git:
git push origin --tags
git tag -n1

exp-200402-test Merge branch 'exp-200402-test' into data
exp-200520-unet Merge branch 'exp-310520-unet' into data
exp-200531-unet Merge branch 'heads/exp-310520-unet' into data
exp-201231-clustsim exp-201231-clustsim
exp-210204-unet Add exp-210204-unet LabBook part 3
exp-310520-unet move exp-310520-unet to data branch manually

1.4 Organization of code

1.4.1 scripts:

1.4.2 src/

1.4.2.1 fluotracify/
  1. imports/
  2. simulations/
  3. training/
  4. applications/
  5. doc/
1.4.2.2 nanosimpy/
  • cloned from dwaithe with refactoring for Python 3-compatibility

1.5 Changes in this repository (without "* Data" in this file)

1.5.1 Changes in LabBook.org (without "* Data")

1.5.1.1 2022-02-19
  • Add #+HTML_HEAD_EXTRA: <style... for table to enable scrolling if the table overflows
1.5.1.2 2021-12-16
  • Add details blocks, corresponding #+HTML_HEAD_EXTRA: <style... and documentation in Notes on archiving
1.5.1.3 2021-08-05
  • Rename master branch to main branch
1.5.1.4 2021-04-04
  • Add #+OPTIONS: H:4 and #+OPTIONS: toc:4 to show up to 4 levels of depth in the html (twbs) export of this LabBook in the table of contents at the side
  • I added Notes on archiving
1.5.1.5 2020-11-04
1.5.1.6 2020-05-31
  • extend general documentation in README
  • Add code block examples
  • extend documentation on experiment workflow
  • move setup notes from README to "Template for data entry and setup notes"
  • remove emacs-lisp code for custom tmux block functions (not relevant enough)
  • change named "jpt-tmux" from starting a jupyter notebook to starting jupyter lab. Load a conda environment instead of using Lmod's module load
1.5.1.7 2020-05-07
  • extend documentation on git model
  • extend documentation on jupyter setup
1.5.1.8 2020-04-22
  • added parts of README which describe the experimental process
  • added templates for system metadata, tmux, jupyter setup
  • added organization of code
1.5.1.9 2020-03-30
  • set up lab book and form git repo accoring to setup by Luka Stanisic et al

1.5.2 Changes in src/fluotracify

2 Data

2.1 exp-210807-hparams

2.1.1 Connect

2.1.1.1 GPU node for script execution
  1. Setup tmux
    rm: cannot remove 'home/lex.tmux-local-socket-remote-machine': No such file or directory
    ye53nis@ara-login01.rz.uni-jena.de's password:              
    /tmp/tmux-67339/default                
    > ye53nis@ara-login01.rz.uni-jena.de's password:            
  2. first, connect with the GPU node in the high performance cluster 
    cd /
srun -p gpu_v100 --time=5-10:00:00 --ntasks-per-node=12 --mem-per-cpu=4000 --gres=gpu:1 --pty bash

    (base) [ye53nis@node130 /]$
  3. Load CUDA and cuDNN in the version compatible to your tensorflow library (see https://www.tensorflow.org/install/source#gpu) 
    module load nvidia/cuda/11.2
module load nvidia/cudnn/8.1
module list
  4. Branch out git branch exp-210807-hparams from main (done via magit) and make sure you are on the correct branch 
    cd /beegfs/ye53nis/drmed-git
git checkout exp-210807-hparams

    (base) [ye53nis@node130 drmed-git]$ git checkout exp-210807-hparams
Checking out files: 100% (148/148), done.
M       src/nanosimpy
Branch exp-210807-hparams set up to track remote branch exp-210807-hparams from origin.
Switched to a new branch 'exp-210807-hparams'
\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting .git/hooks/post-checkout.\n
(base) [ye53nis@node130 drmed-git]$
  5. Side quest: remove last remnants of failed git LFS experiments 
    cat .git/hooks/post-checkout
rm .git/hooks/post-checkout
rm .git/hooks/pre-push

    (base) [ye53nis@node130 drmed-git]$ cat .git/hooks/post-checkout
#!/bin/sh
command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting .git/hooks/post-checkout.\n"; ex
it 2; }
git lfs post-checkout "$@"
(base) [ye53nis@node130 drmed-git]$ rm .git/hooks/post-checkout
(base) [ye53nis@node130 drmed-git]$ rm .git/hooks/pre-push
(base) [ye53nis@node130 drmed-git]$ git push origin exp-210807-hparams
Username for 'https://github.com': aseltmann
Password for 'https://aseltmann@github.com':
Everything up-to-date
(base) [ye53nis@node130 drmed-git]$
  6. Load conda environment 
    conda activate tf

    (tf) [ye53nis@node130 drmed-git]$
  7. define MLflow environment variables and create log directory 
    cd /beegfs/ye53nis/drmed-git
export MLFLOW_EXPERIMENT_NAME=exp-210807-hparams
export MLFLOW_TRACKING_URI=file:/beegfs/ye53nis/drmed-git/data/mlruns
mkdir data/exp-210807-hparams

    (tf) [ye53nis@node130 drmed-git]$
2.1.1.2 Node for running Jupyter
  1. customize the output folder using the following org-mode variable: 
    (setq org-babel-jupyter-resource-directory "./data/exp-210807-hparams/jupyter")

    ./data/exp-210807-hparams/jupyter
  2. Set up tmux (if we haven't done that before) (#+CALL: setup-tmux[:session local])
         
    sh-5.1$ ye53nis@ara-login01.rz.uni-jena.de's password:
    > ye53nis@ara-login01.rz.uni-jena.de's password:
  3. Request compute node 
    cd /
srun -p s_standard --time=7-10:00:00 --ntasks-per-node=24 --mem-per-cpu=2000 --pty bash
  4. Start Jupyter Lab (#+CALL: jpt-tmux[:session jpmux]) 
    (tf) [ye53nis@node144 /]$ jupyter lab --no-browser --port=$PORT
[I 2021-12-14 13:02:25.053 ServerApp] jupyterlab | extension was successfully linked.
[I 2021-12-14 13:02:30.949 ServerApp] nbclassic | extension was successfully linked.
[I 2021-12-14 13:02:31.454 LabApp] JupyterLab extension loaded from /home/ye53nis/.conda/envs/tf/lib/python3.9/site-packages/jupyterlab
[I 2021-12-14 13:02:31.455 LabApp] JupyterLab application directory is /home/ye53nis/.conda/envs/tf/share/jupyter/lab
[I 2021-12-14 13:02:31.464 ServerApp] jupyterlab | extension was successfully loaded.
[I 2021-12-14 13:02:31.591 ServerApp] nbclassic | extension was successfully loaded.
[I 2021-12-14 13:02:31.592 ServerApp] Serving notebooks from local directory: /
[I 2021-12-14 13:02:31.592 ServerApp] Jupyter Server 1.4.1 is running at:
[I 2021-12-14 13:02:31.592 ServerApp] http://localhost:9999/lab?token=d56ed5d84f4a58d80eccfc44527559c44840ce3fd540c1b2
[I 2021-12-14 13:02:31.592 ServerApp]  or http://127.0.0.1:9999/lab?token=d56ed5d84f4a58d80eccfc44527559c44840ce3fd540c1b2
[I 2021-12-14 13:02:31.592 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2021-12-14 13:02:31.625 ServerApp]

    To access the server, open this file in a browser:
        file:///home/ye53nis/.local/share/jupyter/runtime/jpserver-92394-open.html
    Or copy and paste one of these URLs:
        http://localhost:9999/lab?token=d56ed5d84f4a58d80eccfc44527559c44840ce3fd540c1b2
     or http://127.0.0.1:9999/lab?token=d56ed5d84f4a58d80eccfc44527559c44840ce3fd540c1b2
  5. Create SSH Tunnel for jupyter lab to the local computer (e.g. #+CALL: ssh-tunnel(port="9999", node="node152"))
                     
    sh-5.1$ sh-5.1$ ye53nis@ara-login01.rz.uni-jena.de's password:          
    ye53nis@node144's password:              
    Last login: Tue Dec 14 22:39:42 2021 from login01.ara
  6. I started a Python3 kernel using jupyter-server-list-kernels. Then I added the kernel ID to the :PROPERTIES: drawer of this (and following) subtrees. 
    python3           c4f3acce-60c4-489d-922c-407da110fd6a   a few seconds ago    idle       1
  7. Test (#+CALL: jp-metadata(_long='True)): 
    No of CPUs in system: 72
No of CPUs the current process can use: 24
load average: (17.06, 17.03, 18.02)
os.uname():  posix.uname_result(sysname='Linux', nodename='node162', release='3.10.0-957.1.3.el7.x86_64', version='#1 SMP Thu Nov 29 14:49:43 UTC 2018', machine='x86_64')
PID of process: 40969
RAM total: 199G, RAM used: 13G, RAM free: 164G
the current directory: /
My disk usage:
Filesystem           Size  Used Avail Use% Mounted on
/dev/sda1             50G  3.8G   47G   8% /
devtmpfs              94G     0   94G   0% /dev
tmpfs                 94G  8.6M   94G   1% /dev/shm
tmpfs                 94G  115M   94G   1% /run
tmpfs                 94G     0   94G   0% /sys/fs/cgroup
nfs01-ib:/home        80T   67T   14T  83% /home
nfs03-ib:/pool/work  100T   78T   22T  79% /nfsdata
nfs02-ib:/data01      88T   71T   17T  82% /data01
nfs01-ib:/cluster    2.0T  464G  1.6T  23% /cluster
/dev/sda5            2.0G   34M  2.0G   2% /tmp
/dev/sda3            6.0G  428M  5.6G   7% /var
/dev/sda6            169G   33M  169G   1% /local
beegfs_nodev         524T  484T   41T  93% /beegfs
tmpfs                 19G     0   19G   0% /run/user/67339# packages in environment at /home/ye53nis/.conda/envs/tf:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
_openmp_mutex             4.5                       1_gnu
absl-py                   0.13.0                   pypi_0    pypi
alembic                   1.4.1                    pypi_0    pypi
anyio                     2.2.0            py39h06a4308_1
argon2-cffi               20.1.0           py39h27cfd23_1
asteval                   0.9.25                   pypi_0    pypi
astunparse                1.6.3                    pypi_0    pypi
async_generator           1.10               pyhd3eb1b0_0
attrs                     21.2.0             pyhd3eb1b0_0
babel                     2.9.1              pyhd3eb1b0_0
backcall                  0.2.0              pyhd3eb1b0_0
bleach                    3.3.1              pyhd3eb1b0_0
brotlipy                  0.7.0           py39h27cfd23_1003
ca-certificates           2021.7.5             h06a4308_1
cachetools                4.2.2                    pypi_0    pypi
certifi                   2021.5.30        py39h06a4308_0
cffi                      1.14.6           py39h400218f_0
chardet                   4.0.0           py39h06a4308_1003
click                     8.0.1                    pypi_0    pypi
cloudpickle               1.6.0                    pypi_0    pypi
cryptography              3.4.7            py39hd23ed53_0
cycler                    0.10.0                   pypi_0    pypi
databricks-cli            0.14.3                   pypi_0    pypi
decorator                 5.0.9              pyhd3eb1b0_0
defusedxml                0.7.1              pyhd3eb1b0_0
docker                    5.0.0                    pypi_0    pypi
entrypoints               0.3              py39h06a4308_0
fcsfiles                  2021.6.6                 pypi_0    pypi
flask                     2.0.1                    pypi_0    pypi
flatbuffers               1.12                     pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
gast                      0.4.0                    pypi_0    pypi
gitdb                     4.0.7                    pypi_0    pypi
gitpython                 3.1.18                   pypi_0    pypi
google-auth               1.34.0                   pypi_0    pypi
google-auth-oauthlib      0.4.5                    pypi_0    pypi
google-pasta              0.2.0                    pypi_0    pypi
greenlet                  1.1.0                    pypi_0    pypi
grpcio                    1.34.1                   pypi_0    pypi
gunicorn                  20.1.0                   pypi_0    pypi
h5py                      3.1.0                    pypi_0    pypi
idna                      2.10               pyhd3eb1b0_0
importlib-metadata        3.10.0           py39h06a4308_0
importlib_metadata        3.10.0               hd3eb1b0_0
ipykernel                 5.3.4            py39hb070fc8_0
ipython                   7.22.0           py39hb070fc8_0
ipython_genutils          0.2.0              pyhd3eb1b0_1
itsdangerous              2.0.1                    pypi_0    pypi
jedi                      0.17.2           py39h06a4308_1
jinja2                    3.0.1              pyhd3eb1b0_0
joblib                    1.0.1                    pypi_0    pypi
json5                     0.9.6              pyhd3eb1b0_0
jsonschema                3.2.0                      py_2
jupyter-packaging         0.7.12             pyhd3eb1b0_0
jupyter_client            6.1.12             pyhd3eb1b0_0
jupyter_core              4.7.1            py39h06a4308_0
jupyter_server            1.4.1            py39h06a4308_0
jupyterlab                3.0.14             pyhd3eb1b0_1
jupyterlab_pygments       0.1.2                      py_0
jupyterlab_server         2.6.1              pyhd3eb1b0_0
keras-nightly             2.5.0.dev2021032900          pypi_0    pypi
keras-preprocessing       1.1.2                    pypi_0    pypi
kiwisolver                1.3.1                    pypi_0    pypi
ld_impl_linux-64          2.35.1               h7274673_9
libffi                    3.3                  he6710b0_2
libgcc-ng                 9.3.0               h5101ec6_17
libgomp                   9.3.0               h5101ec6_17
libsodium                 1.0.18               h7b6447c_0
libstdcxx-ng              9.3.0               hd4cf53a_17
lmfit                     1.0.2                    pypi_0    pypi
mako                      1.1.4                    pypi_0    pypi
markdown                  3.3.4                    pypi_0    pypi
markupsafe                2.0.1            py39h27cfd23_0
matplotlib                3.4.2                    pypi_0    pypi
mistune                   0.8.4           py39h27cfd23_1000
mlflow                    1.19.0                   pypi_0    pypi
multipletau               0.3.3                    pypi_0    pypi
nbclassic                 0.2.6              pyhd3eb1b0_0
nbclient                  0.5.3              pyhd3eb1b0_0
nbconvert                 6.1.0            py39h06a4308_0
nbformat                  5.1.3              pyhd3eb1b0_0
ncurses                   6.2                  he6710b0_1
nest-asyncio              1.5.1              pyhd3eb1b0_0
notebook                  6.4.0            py39h06a4308_0
numpy                     1.19.5                   pypi_0    pypi
oauthlib                  3.1.1                    pypi_0    pypi
openssl                   1.1.1k               h27cfd23_0
opt-einsum                3.3.0                    pypi_0    pypi
packaging                 21.0               pyhd3eb1b0_0
pandas                    1.3.1                    pypi_0    pypi
pandocfilters             1.4.3            py39h06a4308_1
parso                     0.7.0                      py_0
pexpect                   4.8.0              pyhd3eb1b0_3
pickleshare               0.7.5           pyhd3eb1b0_1003
pillow                    8.3.1                    pypi_0    pypi
pip                       21.1.3           py39h06a4308_0
prometheus-flask-exporter 0.18.2                   pypi_0    pypi
prometheus_client         0.11.0             pyhd3eb1b0_0
prompt-toolkit            3.0.17             pyh06a4308_0
protobuf                  3.17.3                   pypi_0    pypi
ptyprocess                0.7.0              pyhd3eb1b0_2
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pycparser                 2.20                       py_2
pygments                  2.9.0              pyhd3eb1b0_0
pyopenssl                 20.0.1             pyhd3eb1b0_1
pyparsing                 2.4.7              pyhd3eb1b0_0
pyrsistent                0.18.0           py39h7f8727e_0
pysocks                   1.7.1            py39h06a4308_0
python                    3.9.5                h12debd9_4
python-dateutil           2.8.2              pyhd3eb1b0_0
python-editor             1.0.4                    pypi_0    pypi
pytz                      2021.1             pyhd3eb1b0_0
pyyaml                    5.4.1                    pypi_0    pypi
pyzmq                     20.0.0           py39h2531618_1
querystring-parser        1.2.4                    pypi_0    pypi
readline                  8.1                  h27cfd23_0
requests                  2.25.1             pyhd3eb1b0_0
requests-oauthlib         1.3.0                    pypi_0    pypi
rsa                       4.7.2                    pypi_0    pypi
scikit-learn              0.24.2                   pypi_0    pypi
scipy                     1.7.0                    pypi_0    pypi
seaborn                   0.11.1                   pypi_0    pypi
send2trash                1.5.0              pyhd3eb1b0_1
setuptools                52.0.0           py39h06a4308_0
six                       1.15.0                   pypi_0    pypi
smmap                     4.0.0                    pypi_0    pypi
sniffio                   1.2.0            py39h06a4308_1
sqlalchemy                1.4.22                   pypi_0    pypi
sqlite                    3.36.0               hc218d9a_0
sqlparse                  0.4.1                    pypi_0    pypi
tabulate                  0.8.9                    pypi_0    pypi
tensorboard               2.5.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.0                    pypi_0    pypi
tensorflow                2.5.0                    pypi_0    pypi
tensorflow-estimator      2.5.0                    pypi_0    pypi
termcolor                 1.1.0                    pypi_0    pypi
terminado                 0.9.4            py39h06a4308_0
testpath                  0.5.0              pyhd3eb1b0_0
threadpoolctl             2.2.0                    pypi_0    pypi
tifffile                  2021.7.30                pypi_0    pypi
tk                        8.6.10               hbc83047_0
tornado                   6.1              py39h27cfd23_0
traitlets                 5.0.5              pyhd3eb1b0_0
typing-extensions         3.7.4.3                  pypi_0    pypi
tzdata                    2021a                h52ac0ba_0
uncertainties             3.1.6                    pypi_0    pypi
urllib3                   1.26.6             pyhd3eb1b0_1
wcwidth                   0.2.5                      py_0
webencodings              0.5.1            py39h06a4308_1
websocket-client          1.1.0                    pypi_0    pypi
werkzeug                  2.0.1                    pypi_0    pypi
wheel                     0.36.2             pyhd3eb1b0_0
wrapt                     1.12.1                   pypi_0    pypi
xz                        5.2.5                h7b6447c_0
zeromq                    4.3.4                h2531618_0
zipp                      3.5.0              pyhd3eb1b0_0
zlib                      1.2.11               h7b6447c_3

Note: you may need to restart the kernel to use updated packages.
{'SLURM_CHECKPOINT_IMAGE_DIR': '/var/slurm/checkpoint',
 'SLURM_NODELIST': 'node162',
 'SLURM_JOB_NAME': 'bash',
 'XDG_SESSION_ID': '44301',
 'SLURMD_NODENAME': 'node162',
 'SLURM_TOPOLOGY_ADDR': 'node162',
 'SLURM_NTASKS_PER_NODE': '24',
 'HOSTNAME': 'login01',
 'SLURM_PRIO_PROCESS': '0',
 'SLURM_SRUN_COMM_PORT': '34890',
 'SHELL': '/bin/bash',
 'TERM': 'xterm-color',
 'SLURM_JOB_QOS': 'qstand',
 'SLURM_PTY_WIN_ROW': '32',
 'HISTSIZE': '1000',
 'TMPDIR': '/tmp',
 'SLURM_TOPOLOGY_ADDR_PATTERN': 'node',
 'SSH_CLIENT': '10.231.181.128 49370 22',
 'CONDA_SHLVL': '2',
 'CONDA_PROMPT_MODIFIER': '(tf) ',
 'QTDIR': '/usr/lib64/qt-3.3',
 'QTINC': '/usr/lib64/qt-3.3/include',
 'SSH_TTY': '/dev/pts/79',
 'NO_PROXY': 'localhost,127.0.0.0/8,.uni-jena.de,141.35.0.0/16,10.0.0.0/8,192.168.0.0/16,172.0.0.0/8,fe80::/7,2001:638:1558::/24,vmaster,node001',
 'QT_GRAPHICSSYSTEM_CHECKED': '1',
 'SLURM_NNODES': '1',
 'USER': 'ye53nis',
 'http_proxy': 'http://internet4nzm.rz.uni-jena.de:3128',
 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:',
 'CONDA_EXE': '/cluster/miniconda3/bin/conda',
 'SLURM_STEP_NUM_NODES': '1',
 'SLURM_JOBID': '1547254',
 'SRUN_DEBUG': '3',
 'FTP_PROXY': 'http://internet4nzm.rz.uni-jena.de:3128',
 'ftp_proxy': 'http://internet4nzm.rz.uni-jena.de:3128',
 'SLURM_NTASKS': '24',
 'SLURM_LAUNCH_NODE_IPADDR': '192.168.192.5',
 'SLURM_STEP_ID': '0',
 'TMUX': '/tmp/tmux-67339/default,20557,4',
 '_CE_CONDA': '',
 'CONDA_PREFIX_1': '/cluster/miniconda3',
 'SLURM_STEP_LAUNCHER_PORT': '34890',
 'SLURM_TASKS_PER_NODE': '24',
 'MAIL': '/var/spool/mail/ye53nis',
 'PATH': '/home/ye53nis/.conda/envs/tf/bin:/home/lex/Programme/miniconda3/envs/tf-nightly-lab/bin:/home/lex/Programme/miniconda3/condabin:/home/lex/.local/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/var/lib/snapd/snap/bin:/usr/sbin:/home/ye53nis/.local/bin:/home/ye53nis/bin',
 'SLURM_WORKING_CLUSTER': 'hpc:192.168.192.1:6817:8448',
 'SLURM_JOB_ID': '1547254',
 'CONDA_PREFIX': '/home/ye53nis/.conda/envs/tf',
 'SLURM_JOB_USER': 'ye53nis',
 'SLURM_STEPID': '0',
 'PWD': '/',
 'SLURM_SRUN_COMM_HOST': '192.168.192.5',
 'LANG': 'en_US.UTF-8',
 'SLURM_PTY_WIN_COL': '235',
 'SLURM_UMASK': '0022',
 'MODULEPATH': '/usr/share/Modules/modulefiles:/etc/modulefiles:/cluster/modulefiles',
 'SLURM_JOB_UID': '67339',
 'LOADEDMODULES': '',
 'SLURM_NODEID': '0',
 'TMUX_PANE': '%4',
 'SLURM_SUBMIT_DIR': '/',
 'SLURM_TASK_PID': '37261',
 'SLURM_NPROCS': '24',
 'SLURM_CPUS_ON_NODE': '24',
 'SLURM_DISTRIBUTION': 'block',
 'HTTPS_PROXY': 'http://internet4nzm.rz.uni-jena.de:3128',
 'https_proxy': 'http://internet4nzm.rz.uni-jena.de:3128',
 'SLURM_PROCID': '0',
 'HISTCONTROL': 'ignoredups',
 '_CE_M': '',
 'SLURM_JOB_NODELIST': 'node162',
 'SLURM_PTY_PORT': '39060',
 'HOME': '/home/ye53nis',
 'SHLVL': '3',
 'SLURM_LOCALID': '0',
 'SLURM_JOB_GID': '13280',
 'SLURM_JOB_CPUS_PER_NODE': '24',
 'SLURM_CLUSTER_NAME': 'hpc',
 'no_proxy': 'localhost,127.0.0.0/8,.uni-jena.de,141.35.0.0/16,10.0.0.0/8,192.168.0.0/16,172.0.0.0/8,fe80::/7,2001:638:1558::/24,vmaster,node001',
 'SLURM_GTIDS': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23',
 'SLURM_SUBMIT_HOST': 'login01',
 'HTTP_PROXY': 'http://internet4nzm.rz.uni-jena.de:3128',
 'SLURM_JOB_PARTITION': 's_standard',
 'MATHEMATICA_HOME': '/cluster/apps/mathematica/11.3',
 'CONDA_PYTHON_EXE': '/cluster/miniconda3/bin/python',
 'LOGNAME': 'ye53nis',
 'SLURM_STEP_NUM_TASKS': '24',
 'QTLIB': '/usr/lib64/qt-3.3/lib',
 'SLURM_JOB_ACCOUNT': 'iaob',
 'SLURM_JOB_NUM_NODES': '1',
 'MODULESHOME': '/usr/share/Modules',
 'CONDA_DEFAULT_ENV': 'tf',
 'LESSOPEN': '||/usr/bin/lesspipe.sh %s',
 'SLURM_STEP_TASKS_PER_NODE': '24',
 'PORT': '9999',
 'SLURM_STEP_NODELIST': 'node162',
 'DISPLAY': ':0',
 'XDG_RUNTIME_DIR': '',
 'XAUTHORITY': '/home/lex/.Xauthority',
 'BASH_FUNC_module()': '() {  eval `/usr/bin/modulecmd bash $*`\n}',
 '_': '/home/ye53nis/.conda/envs/tf/bin/jupyter',
 'JPY_PARENT_PID': '38132',
 'CLICOLOR': '1',
 'PAGER': 'cat',
 'GIT_PAGER': 'cat',
 'MPLBACKEND': 'module://ipykernel.pylab.backend_inline'}
2.1.1.3 Node for running Mlflow UI
  1. Create mlflow tmux session and start mlflow ui 
    conda activate tf
mlflow ui --backend-store-uri file:///beegfs/ye53nis/drmed-git/data/mlruns -p 5001

    (tf) [ye53nis@login01 ~]$ mlflow ui --backend-store-uri file:///beegfs/ye53nis/drmed-git/data/mlruns -p 5001
[2021-08-08 14:47:33 +0200] [5106] [INFO] Starting gunicorn 20.1.0
[2021-08-08 14:47:33 +0200] [5106] [INFO] Listening at: http://127.0.0.1:5001 (5106)
[2021-08-08 14:47:33 +0200] [5106] [INFO] Using worker: sync
[2021-08-08 14:47:33 +0200] [5115] [INFO] Booting worker with pid: 5115
  2. SHH tunnel the mflow session to the local computer (#+CALL: ssh-tunnel[:session local3](port="5001", node="login01"))
                     
    sh-5.1$ sh-5.1$ ye53nis@ara-login01.rz.uni-jena.de's password:          
    ye53nis@login01's password:              
    bind: Address already in use        
    Last login: Tue Aug 17 18:03:52 2021 from 10.231.188.20

2.1.2 Run 1 - hparams

2.1.2.1 Record metadata
  1. Current directory, last 5 git commits 
    pwd
git log -5

    (tf) [ye53nis@node130 drmed-git]$ pwd
/beegfs/ye53nis/drmed-git
(tf) [ye53nis@node130 drmed-git]$ git log -5
commit aa5b9bc35c53c4fd1525c6b812b2a28532ae7afb
Author: Apoplex <oligolex@vivaldi.net>
Date:   Sat Aug 7 22:15:02 2021 +0200

    Add hparams combi restriction; add metadata

    problems arise, if in the random combination of hparams, 2*pool_size**n_levels
    is bigger than the input_size. That's why these cases are skipped now.

commit 36bfdd79e78f84fe2f05a11d791c33b9c724b71f
Author: Apoplex <oligolex@vivaldi.net>
Date:   Sat Aug 7 14:28:28 2021 +0200

    add drop_remainder, add hparams, remove steps

commit f3f5310a32a110793d016c2c788b1058bbc5439e
Author: Apoplex <oligolex@vivaldi.net>
Date:   Sat Aug 7 14:02:57 2021 +0200

    Change conda environment

commit fcc80acbc07dcc92b207b034f4b4133f2800d0f3
Author: Apoplex <oligolex@vivaldi.net>
Date:   Thu Aug 5 22:16:18 2021 +0200

    Rename master to main

commit 8edc3c254d26b5c18fb262e73c1fd26406b97573
Author: Apoplex <oligolex@vivaldi.net>
Date:   Wed Aug 4 22:36:06 2021 +0200

    Fix mlflow best run logging
(tf) [ye53nis@node130 drmed-git]$
  2. GPU, CPU, RAM, file system, env variables, top info 
    nvcc -V
echo --------------------
lscpu
echo --------------------
nproc
echo --------------------
free -h
echo --------------------
df -h
echo --------------------
printenv
echo --------------------
top -bcn1 -w512 | head -n 15

    (tf) [ye53nis@node130 drmed-git]$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Nov_30_19:08:53_PST_2020
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0

--------------------
(tf) [ye53nis@node130 drmed-git]$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                48
On-line CPU(s) list:   0-47
Thread(s) per core:    2
Core(s) per socket:    12
Socket(s):             2
NUMA node(s):          4
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 79
Model name:            Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
Stepping:              1
CPU MHz:               1200.439
CPU max MHz:           2900.0000
CPU min MHz:           1200.0000
BogoMIPS:              4399.92
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              15360K
NUMA node0 CPU(s):     0-5,24-29
NUMA node1 CPU(s):     6-11,30-35
NUMA node2 CPU(s):     12-17,36-41
NUMA node3 CPU(s):     18-23,42-47
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arc
h_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic m
ovbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb cat_l3 cdp_l3 intel_ppin intel_pt ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_
adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts spec_ctrl intel_stibp

--------------------
(tf) [ye53nis@node130 drmed-git]$ nproc
12

--------------------
(tf) [ye53nis@node130 drmed-git]$ free -h
              total        used        free      shared  buff/cache   available
Mem:           125G        1.2G         78G        229M         45G        122G
Swap:           11G        2.2M         11G

--------------------
(tf) [ye53nis@node130 drmed-git]$ df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/sda1             50G  6.2G   44G  13% /
devtmpfs              63G     0   63G   0% /dev
tmpfs                 63G  180M   63G   1% /dev/shm
tmpfs                 63G   51M   63G   1% /run
tmpfs                 63G     0   63G   0% /sys/fs/cgroup
nfs03-ib:/pool/work  100T   78T   23T  78% /nfsdata
nfs01-ib:/home        80T   62T   19T  77% /home
nfs01-ib:/cluster    2.0T  435G  1.6T  22% /cluster
/dev/sda6            169G  122M  169G   1% /local
/dev/sda5            2.0G   35M  2.0G   2% /tmp
/dev/sda3            6.0G  666M  5.4G  11% /var
beegfs_nodev         524T  432T   92T  83% /beegfs

--------------------
(tf) [ye53nis@node130 drmed-git]$ printenv
SLURM_CHECKPOINT_IMAGE_DIR=/var/slurm/checkpoint
SLURM_NODELIST=node130
CUDA_PATH=/cluster/nvidia/cuda/11.2
SLURM_JOB_NAME=bash
CUDA_INC_PATH=/cluster/nvidia/cuda/11.2/include
XDG_SESSION_ID=44301
SLURMD_NODENAME=node130
SLURM_TOPOLOGY_ADDR=node130
SLURM_NTASKS_PER_NODE=12
HOSTNAME=login01
SLURM_PRIO_PROCESS=0
SLURM_SRUN_COMM_PORT=37107
SHELL=/bin/bash
TERM=screen
MLFLOW_EXPERIMENT_NAME=exp-210807-hparams
SLURM_JOB_QOS=qstand
SLURM_PTY_WIN_ROW=24
HISTSIZE=1000
TMPDIR=/tmp
SLURM_TOPOLOGY_ADDR_PATTERN=node
SSH_CLIENT=10.231.181.128 49370 22
INCLUDEDIR=/cluster/nvidia/cuda/11.2/include
CONDA_SHLVL=2
CONDA_PROMPT_MODIFIER=(tf)
OLDPWD=/beegfs/ye53nis/drmed-git/data
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/79
NO_PROXY=localhost,127.0.0.0/8,.uni-jena.de,141.35.0.0/16,10.0.0.0/8,192.168.0.0/16,172.0.0.0/8,fe80::/7,2001:638:1558::/24,vmaster,node001
QT_GRAPHICSSYSTEM_CHECKED=1
SLURM_NNODES=1
USER=ye53nis
http_proxy=http://internet4nzm.rz.uni-jena.de:3128
LD_LIBRARY_PATH=/cluster/nvidia/cuda/11.2/lib64:/cluster/nvidia/cuda/11.2/nvvm/lib64:/cluster/nvidia/cudnn/8.1//lib64
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01
;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01
;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01
;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tg
a=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01
;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl
=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*
.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
CONDA_EXE=/cluster/miniconda3/bin/conda
SLURM_STEP_NUM_NODES=1
SLURM_JOBID=1461501
SRUN_DEBUG=3
FTP_PROXY=http://internet4nzm.rz.uni-jena.de:3128
ftp_proxy=http://internet4nzm.rz.uni-jena.de:3128
SLURM_NTASKS=12
SLURM_LAUNCH_NODE_IPADDR=192.168.192.5
SLURM_STEP_ID=0
TMUX=/tmp/tmux-67339/default,20557,6
_CE_CONDA=
CONDA_PREFIX_1=/cluster/miniconda3
MODCUDA=YES
SLURM_STEP_LAUNCHER_PORT=37107
SLURM_TASKS_PER_NODE=12
MAIL=/var/spool/mail/ye53nis
PATH=/cluster/nvidia/cuda/11.2/bin:/cluster/nvidia/cuda/11.2/nvvm:/cluster/nvidia/cuda/11.2/open64/bin:/cluster/nvidia/cuda/11.2/libnvvp:/home/ye53nis/.conda/envs/tf/bin:/home/lex/Programm
e/miniconda3/envs/tf-nightly-lab/bin:/home/lex/Programme/miniconda3/condabin:/home/lex/.local/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/
usr/bin/vendor_perl:/usr/bin/core_perl:/var/lib/snapd/snap/bin:/usr/sbin:/home/ye53nis/.local/bin:/home/ye53nis/bin
SLURM_WORKING_CLUSTER=hpc:192.168.192.1:6817:8448
SLURM_JOB_ID=1461501
LD_RUN_PATH=/cluster/nvidia/cuda/11.2/lib64
SLURM_STEP_GPUS=0
CONDA_PREFIX=/home/ye53nis/.conda/envs/tf
CUDA_LIB_PATH=/cluster/nvidia/cuda/11.2/lib64
SLURM_JOB_USER=ye53nis
SLURM_STEPID=0
PWD=/beegfs/ye53nis/drmed-git
_LMFILES_=/cluster/modulefiles/nvidia/cuda/11.2:/cluster/modulefiles/nvidia/cudnn/8.1
CUDA_VISIBLE_DEVICES=0
SLURM_SRUN_COMM_HOST=192.168.192.5
LANG=en_US.UTF-8
SLURM_PTY_WIN_COL=80
SLURM_UMASK=0022
MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/cluster/modulefiles
SLURM_JOB_UID=67339
LOADEDMODULES=nvidia/cuda/11.2:nvidia/cudnn/8.1
SLURM_NODEID=0
TMUX_PANE=%6
SLURM_SUBMIT_DIR=/
SLURM_TASK_PID=11596
SLURM_NPROCS=12
SLURM_CPUS_ON_NODE=12
SLURM_DISTRIBUTION=block
HTTPS_PROXY=http://internet4nzm.rz.uni-jena.de:3128
https_proxy=http://internet4nzm.rz.uni-jena.de:3128
SLURM_PROCID=0
HISTCONTROL=ignoredups
_CE_M=
SLURM_JOB_NODELIST=node130
SLURM_PTY_PORT=41921
HOME=/home/ye53nis
SHLVL=3
SLURM_LOCALID=0
SLURM_JOB_GID=13280
SLURM_JOB_CPUS_PER_NODE=12
SLURM_CLUSTER_NAME=hpc
no_proxy=localhost,127.0.0.0/8,.uni-jena.de,141.35.0.0/16,10.0.0.0/8,192.168.0.0/16,172.0.0.0/8,fe80::/7,2001:638:1558::/24,vmaster,node001
SLURM_GTIDS=0,1,2,3,4,5,6,7,8,9,10,11
SLURM_SUBMIT_HOST=login01
HTTP_PROXY=http://internet4nzm.rz.uni-jena.de:3128
SLURM_JOB_PARTITION=gpu_v100
MATHEMATICA_HOME=/cluster/apps/mathematica/11.3
CONDA_PYTHON_EXE=/cluster/miniconda3/bin/python
LOGNAME=ye53nis
SLURM_STEP_NUM_TASKS=12
QTLIB=/usr/lib64/qt-3.3/lib
GPU_DEVICE_ORDINAL=0
SLURM_JOB_ACCOUNT=iaob
MLFLOW_TRACKING_URI=file:/beegfs/ye53nis/drmed-git/data/mlruns
SLURM_JOB_NUM_NODES=1
MODULESHOME=/usr/share/Modules
CONDA_DEFAULT_ENV=tf
LESSOPEN=||/usr/bin/lesspipe.sh %s
SLURM_STEP_TASKS_PER_NODE=12
SLURM_STEP_NODELIST=node130
DISPLAY=:0
XDG_RUNTIME_DIR=/run/user/67339
INCLUDE=/cluster/nvidia/cudnn/8.1//include
XAUTHORITY=/home/lex/.Xauthority
BASH_FUNC_module()=() {  eval `/usr/bin/modulecmd bash $*`
}
_=/bin/printenv

--------------------
(tf) [ye53nis@node130 drmed-git]$ top -bcn1 -w512 | head -n 15
top - 15:07:22 up 89 days, 23:51,  0 users,  load average: 0.00, 0.01, 0.05
Tasks: 513 total,   2 running, 511 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.2 us,  0.2 sy,  0.0 ni, 99.6 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 13191629+total, 82832160 free,  1227056 used, 47857076 buff/cache
KiB Swap: 12582908 total, 12580604 free,     2304 used. 12863139+avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 9574 ye53nis   20   0  172600   2616   1668 R  11.8  0.0   0:00.03 top -bcn1 -w512
    1 root      20   0   71816   7552   2584 S   0.0  0.0  38:23.65 /usr/lib/systemd/systemd --switched-root --system --deserialize 22
    2 root      20   0       0      0      0 S   0.0  0.0   0:05.31 [kthreadd]
    3 root      20   0       0      0      0 S   0.0  0.0   2:30.24 [ksoftirqd/0]
    5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 [kworker/0:0H]
    8 root      rt   0       0      0      0 S   0.0  0.0   0:43.26 [migration/0]
    9 root      20   0       0      0      0 S   0.0  0.0   0:00.00 [rcu_bh]
   10 root      20   0       0      0      0 R   0.0  0.0 275:37.95 [rcu_sched]
(tf) [ye53nis@node130 drmed-git]$
  3. print conda list 
    conda list

    (tf) [ye53nis@node130 drmed-git]$ conda list
# packages in environment at /home/ye53nis/.conda/envs/tf:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
_openmp_mutex             4.5                       1_gnu
absl-py                   0.13.0                   pypi_0    pypi
alembic                   1.4.1                    pypi_0    pypi
anyio                     2.2.0            py39h06a4308_1
argon2-cffi               20.1.0           py39h27cfd23_1
asteval                   0.9.25                   pypi_0    pypi
astunparse                1.6.3                    pypi_0    pypi
async_generator           1.10               pyhd3eb1b0_0
attrs                     21.2.0             pyhd3eb1b0_0
babel                     2.9.1              pyhd3eb1b0_0
backcall                  0.2.0              pyhd3eb1b0_0
bleach                    3.3.1              pyhd3eb1b0_0
brotlipy                  0.7.0           py39h27cfd23_1003
ca-certificates           2021.7.5             h06a4308_1
cachetools                4.2.2                    pypi_0    pypi
certifi                   2021.5.30        py39h06a4308_0
cffi                      1.14.6           py39h400218f_0
chardet                   4.0.0           py39h06a4308_1003
click                     8.0.1                    pypi_0    pypi
cloudpickle               1.6.0                    pypi_0    pypi
cryptography              3.4.7            py39hd23ed53_0
cycler                    0.10.0                   pypi_0    pypi
databricks-cli            0.14.3                   pypi_0    pypi
decorator                 5.0.9              pyhd3eb1b0_0
defusedxml                0.7.1              pyhd3eb1b0_0
docker                    5.0.0                    pypi_0    pypi
entrypoints               0.3              py39h06a4308_0
fcsfiles                  2021.6.6                 pypi_0    pypi
flask                     2.0.1                    pypi_0    pypi
flatbuffers               1.12                     pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
gast                      0.4.0                    pypi_0    pypi
gitdb                     4.0.7                    pypi_0    pypi
gitpython                 3.1.18                   pypi_0    pypi
google-auth               1.34.0                   pypi_0    pypi
google-auth-oauthlib      0.4.5                    pypi_0    pypi
google-pasta              0.2.0                    pypi_0    pypi
greenlet                  1.1.0                    pypi_0    pypi
grpcio                    1.34.1                   pypi_0    pypi
gunicorn                  20.1.0                   pypi_0    pypi
h5py                      3.1.0                    pypi_0    pypi
idna                      2.10               pyhd3eb1b0_0
importlib-metadata        3.10.0           py39h06a4308_0
importlib_metadata        3.10.0               hd3eb1b0_0
ipykernel                 5.3.4            py39hb070fc8_0
ipython                   7.22.0           py39hb070fc8_0
ipython_genutils          0.2.0              pyhd3eb1b0_1
itsdangerous              2.0.1                    pypi_0    pypi
jedi                      0.17.2           py39h06a4308_1
jinja2                    3.0.1              pyhd3eb1b0_0
joblib                    1.0.1                    pypi_0    pypi
json5                     0.9.6              pyhd3eb1b0_0
jsonschema                3.2.0                      py_2
jupyter-packaging         0.7.12             pyhd3eb1b0_0
jupyter_client            6.1.12             pyhd3eb1b0_0
jupyter_core              4.7.1            py39h06a4308_0
jupyter_server            1.4.1            py39h06a4308_0
jupyterlab                3.0.14             pyhd3eb1b0_1
jupyterlab_pygments       0.1.2                      py_0
jupyterlab_server         2.6.1              pyhd3eb1b0_0
keras-nightly             2.5.0.dev2021032900          pypi_0    pypi
keras-preprocessing       1.1.2                    pypi_0    pypi
kiwisolver                1.3.1                    pypi_0    pypi
ld_impl_linux-64          2.35.1               h7274673_9
libffi                    3.3                  he6710b0_2
libgcc-ng                 9.3.0               h5101ec6_17
libgomp                   9.3.0               h5101ec6_17
libsodium                 1.0.18               h7b6447c_0
libstdcxx-ng              9.3.0               hd4cf53a_17
lmfit                     1.0.2                    pypi_0    pypi
mako                      1.1.4                    pypi_0    pypi
markdown                  3.3.4                    pypi_0    pypi
markupsafe                2.0.1            py39h27cfd23_0
matplotlib                3.4.2                    pypi_0    pypi
mistune                   0.8.4           py39h27cfd23_1000
mlflow                    1.19.0                   pypi_0    pypi
multipletau               0.3.3                    pypi_0    pypi
nbclassic                 0.2.6              pyhd3eb1b0_0
nbclient                  0.5.3              pyhd3eb1b0_0
nbconvert                 6.1.0            py39h06a4308_0
nbformat                  5.1.3              pyhd3eb1b0_0
ncurses                   6.2                  he6710b0_1
nest-asyncio              1.5.1              pyhd3eb1b0_0
notebook                  6.4.0            py39h06a4308_0
numpy                     1.19.5                   pypi_0    pypi
oauthlib                  3.1.1                    pypi_0    pypi
openssl                   1.1.1k               h27cfd23_0
opt-einsum                3.3.0                    pypi_0    pypi
packaging                 21.0               pyhd3eb1b0_0
pandas                    1.3.1                    pypi_0    pypi
pandocfilters             1.4.3            py39h06a4308_1
parso                     0.7.0                      py_0
pexpect                   4.8.0              pyhd3eb1b0_3
pickleshare               0.7.5           pyhd3eb1b0_1003
pillow                    8.3.1                    pypi_0    pypi
pip                       21.1.3           py39h06a4308_0
prometheus-flask-exporter 0.18.2                   pypi_0    pypi
prometheus_client         0.11.0             pyhd3eb1b0_0
prompt-toolkit            3.0.17             pyh06a4308_0
protobuf                  3.17.3                   pypi_0    pypi
ptyprocess                0.7.0              pyhd3eb1b0_2
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pycparser                 2.20                       py_2
pygments                  2.9.0              pyhd3eb1b0_0
pyopenssl                 20.0.1             pyhd3eb1b0_1
pyparsing                 2.4.7              pyhd3eb1b0_0
pyrsistent                0.18.0           py39h7f8727e_0
pysocks                   1.7.1            py39h06a4308_0
python                    3.9.5                h12debd9_4
python-dateutil           2.8.2              pyhd3eb1b0_0
python-editor             1.0.4                    pypi_0    pypi
pytz                      2021.1             pyhd3eb1b0_0
pyyaml                    5.4.1                    pypi_0    pypi
pyzmq                     20.0.0           py39h2531618_1
querystring-parser        1.2.4                    pypi_0    pypi
readline                  8.1                  h27cfd23_0
requests                  2.25.1             pyhd3eb1b0_0
requests-oauthlib         1.3.0                    pypi_0    pypi
rsa                       4.7.2                    pypi_0    pypi
scikit-learn              0.24.2                   pypi_0    pypi
scipy                     1.7.0                    pypi_0    pypi
seaborn                   0.11.1                   pypi_0    pypi
send2trash                1.5.0              pyhd3eb1b0_1
setuptools                52.0.0           py39h06a4308_0
six                       1.15.0                   pypi_0    pypi
smmap                     4.0.0                    pypi_0    pypi
sniffio                   1.2.0            py39h06a4308_1
sqlalchemy                1.4.22                   pypi_0    pypi
sqlite                    3.36.0               hc218d9a_0
sqlparse                  0.4.1                    pypi_0    pypi
tabulate                  0.8.9                    pypi_0    pypi
tensorboard               2.5.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.0                    pypi_0    pypi
tensorflow                2.5.0                    pypi_0    pypi
tensorflow-estimator      2.5.0                    pypi_0    pypi
termcolor                 1.1.0                    pypi_0    pypi
terminado                 0.9.4            py39h06a4308_0
testpath                  0.5.0              pyhd3eb1b0_0
threadpoolctl             2.2.0                    pypi_0    pypi
tifffile                  2021.7.30                pypi_0    pypi
tk                        8.6.10               hbc83047_0
tornado                   6.1              py39h27cfd23_0
traitlets                 5.0.5              pyhd3eb1b0_0
typing-extensions         3.7.4.3                  pypi_0    pypi
tzdata                    2021a                h52ac0ba_0
uncertainties             3.1.6                    pypi_0    pypi
urllib3                   1.26.6             pyhd3eb1b0_1
wcwidth                   0.2.5                      py_0
webencodings              0.5.1            py39h06a4308_1
websocket-client          1.1.0                    pypi_0    pypi
werkzeug                  2.0.1                    pypi_0    pypi
wheel                     0.36.2             pyhd3eb1b0_0
wrapt                     1.12.1                   pypi_0    pypi
xz                        5.2.5                h7b6447c_0
zeromq                    4.3.4                h2531618_0
zipp                      3.5.0              pyhd3eb1b0_0
zlib                      1.2.11               h7b6447c_3
(tf) [ye53nis@node130 drmed-git]$
  4. Show tree of input files used. I made a design choice in simulating these files, which I now regret: I simulated 10 files for each speed of molecules (0.069, 0.08, 0.1, …), but I randomly chose the clusters. In hindsight, the cluster speed creates vastly different artifact shapes, which may have implications for the training. Thats why I manually created different splits of train and val data. 
    tree ../saves/firstartifact_Nov2020_train_max3sets
echo --------------------
tree ../saves/firstartifact_Nov2020_val_max3sets
echo --------------------
tree ../saves/firstartifact_Nov2020_train_max2sets
echo --------------------
tree ../saves/firstartifact_Nov2020_val_max2sets_SORTEDIN
echo --------------------
tree ../saves/firstartifact_Nov2020_train_max1set
echo --------------------
tree ../saves/firstartifact_Nov2020_test

    ../saves/firstartifact_Nov2020_train_max3sets
├── 0.069
│   ├── 0.01
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.069_set002.csv
│   │   ├── traces_brightclust_Nov2020_D0.069_set003.csv
│   │   └── traces_brightclust_Nov2020_D0.069_set006.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.069_set009.csv
├── 0.08
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.08_set007.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.08_set002.csv
│   │   ├── traces_brightclust_Nov2020_D0.08_set006.csv
│   │   └── traces_brightclust_Nov2020_D0.08_set008.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.08_set004.csv
│       └── traces_brightclust_Nov2020_D0.08_set009.csv
├── 0.1
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D0.1_set004.csv
│   │   ├── traces_brightclust_Nov2020_D0.1_set006.csv
│   │   └── traces_brightclust_Nov2020_D0.1_set008.csv
│   ├── 0.1
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.1_set003.csv
│       └── traces_brightclust_Nov2020_D0.1_set007.csv
├── 0.2
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.2_set003.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.2_set001.csv
│   │   ├── traces_brightclust_Nov2020_D0.2_set004.csv
│   │   └── traces_brightclust_Nov2020_D0.2_set006.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.2_set009.csv
│       └── traces_brightclust_Nov2020_D0.2_set010.csv
├── 0.4
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D0.4_set004.csv
│   │   └── traces_brightclust_Nov2020_D0.4_set010.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.4_set002.csv
│   │   ├── traces_brightclust_Nov2020_D0.4_set003.csv
│   │   └── traces_brightclust_Nov2020_D0.4_set009.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.4_set006.csv
│       └── traces_brightclust_Nov2020_D0.4_set007.csv
├── 0.6
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.6_set010.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.6_set004.csv
│   │   ├── traces_brightclust_Nov2020_D0.6_set005.csv
│   │   └── traces_brightclust_Nov2020_D0.6_set006.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.6_set001.csv
│       └── traces_brightclust_Nov2020_D0.6_set002.csv
├── 10
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D10_set003.csv
│   │   ├── traces_brightclust_Nov2020_D10_set004.csv
│   │   └── traces_brightclust_Nov2020_D10_set008.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D10_set006.csv
│   │   └── traces_brightclust_Nov2020_D10_set007.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D10_set010.csv
├── 1.0
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D1.0_set010.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D1.0_set004.csv
│   │   ├── traces_brightclust_Nov2020_D1.0_set007.csv
│   │   └── traces_brightclust_Nov2020_D1.0_set009.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D1.0_set001.csv
│       ├── traces_brightclust_Nov2020_D1.0_set002.csv
│       └── traces_brightclust_Nov2020_D1.0_set008.csv
├── 3.0
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D3.0_set005.csv
│   │   ├── traces_brightclust_Nov2020_D3.0_set006.csv
│   │   └── traces_brightclust_Nov2020_D3.0_set008.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D3.0_set010.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D3.0_set001.csv
│       ├── traces_brightclust_Nov2020_D3.0_set003.csv
│       └── traces_brightclust_Nov2020_D3.0_set009.csv
└── 50
    ├── 0.01
    │   └── traces_brightclust_Nov2020_D50_set006.csv
    ├── 0.1
    │   ├── traces_brightclust_Nov2020_D50_set009.csv
    │   └── traces_brightclust_Nov2020_D50_set010.csv
    └── 1.0
        ├── traces_brightclust_Nov2020_D50_set004.csv
        ├── traces_brightclust_Nov2020_D50_set005.csv
        └── traces_brightclust_Nov2020_D50_set007.csv

40 directories, 60 files

--------------------
../saves/firstartifact_Nov2020_val_max3sets
├── 0.069
│   ├── 0.01
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.069_set007.csv
│   │   └── traces_brightclust_Nov2020_D0.069_set008.csv
│   └── 1.0
├── 0.08
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.08_set010.csv
│   └── 1.0
├── 0.1
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D0.1_set009.csv
│   │   └── traces_brightclust_Nov2020_D0.1_set010.csv
│   ├── 0.1
│   └── 1.0
├── 0.2
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.2_set008.csv
│   └── 1.0
├── 0.4
│   ├── 0.01
│   ├── 0.1
│   └── 1.0
├── 0.6
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.6_set007.csv
│   └── 1.0
├── 10
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D10_set009.csv
│   ├── 0.1
│   └── 1.0
├── 1.0
│   ├── 0.01
│   ├── 0.1
│   └── 1.0
├── 3.0
│   ├── 0.01
│   ├── 0.1
│   └── 1.0
└── 50
    ├── 0.01
    ├── 0.1
    └── 1.0
        └── traces_brightclust_Nov2020_D50_set008.csv

40 directories, 9 files

--------------------
../saves/firstartifact_Nov2020_train_max2sets
├── 0.069
│   ├── 0.01
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.069_set002.csv
│   │   └── traces_brightclust_Nov2020_D0.069_set003.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.069_set009.csv
├── 0.08
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.08_set007.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.08_set002.csv
│   │   └── traces_brightclust_Nov2020_D0.08_set006.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.08_set004.csv
│       └── traces_brightclust_Nov2020_D0.08_set009.csv
├── 0.1
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D0.1_set004.csv
│   │   └── traces_brightclust_Nov2020_D0.1_set006.csv
│   ├── 0.1
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.1_set003.csv
│       └── traces_brightclust_Nov2020_D0.1_set007.csv
├── 0.2
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.2_set003.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.2_set001.csv
│   │   └── traces_brightclust_Nov2020_D0.2_set004.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.2_set009.csv
│       └── traces_brightclust_Nov2020_D0.2_set010.csv
├── 0.4
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D0.4_set004.csv
│   │   └── traces_brightclust_Nov2020_D0.4_set010.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.4_set002.csv
│   │   └── traces_brightclust_Nov2020_D0.4_set003.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.4_set006.csv
│       └── traces_brightclust_Nov2020_D0.4_set007.csv
├── 0.6
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.6_set010.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D0.6_set004.csv
│   │   └── traces_brightclust_Nov2020_D0.6_set005.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D0.6_set001.csv
│       └── traces_brightclust_Nov2020_D0.6_set002.csv
├── 10
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D10_set003.csv
│   │   └── traces_brightclust_Nov2020_D10_set004.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D10_set006.csv
│   │   └── traces_brightclust_Nov2020_D10_set007.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D10_set010.csv
├── 1.0
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D1.0_set010.csv
│   ├── 0.1
│   │   ├── traces_brightclust_Nov2020_D1.0_set004.csv
│   │   └── traces_brightclust_Nov2020_D1.0_set007.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D1.0_set001.csv
│       └── traces_brightclust_Nov2020_D1.0_set002.csv
├── 3.0
│   ├── 0.01
│   │   ├── traces_brightclust_Nov2020_D3.0_set005.csv
│   │   └── traces_brightclust_Nov2020_D3.0_set006.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D3.0_set010.csv
│   └── 1.0
│       ├── traces_brightclust_Nov2020_D3.0_set001.csv
│       └── traces_brightclust_Nov2020_D3.0_set003.csv
└── 50
    ├── 0.01
    │   └── traces_brightclust_Nov2020_D50_set006.csv
    ├── 0.1
    │   ├── traces_brightclust_Nov2020_D50_set009.csv
    │   └── traces_brightclust_Nov2020_D50_set010.csv
    └── 1.0
        ├── traces_brightclust_Nov2020_D50_set004.csv
        └── traces_brightclust_Nov2020_D50_set005.csv

40 directories, 48 files

--------------------
../saves/firstartifact_Nov2020_val_max2sets_SORTEDIN
├── 0.069
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.069_set006.csv
│   └── 1.0
├── 0.08
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.08_set008.csv
│   └── 1.0
├── 0.1
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.1_set008.csv
│   ├── 0.1
│   └── 1.0
├── 0.2
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.2_set006.csv
│   └── 1.0
├── 0.4
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.4_set009.csv
│   └── 1.0
├── 0.6
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.6_set006.csv
│   └── 1.0
├── 10
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D10_set008.csv
│   ├── 0.1
│   └── 1.0
├── 1.0
│   ├── 0.01
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D1.0_set009.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D1.0_set008.csv
├── 3.0
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D3.0_set008.csv
│   ├── 0.1
│   └── 1.0
│       └── traces_brightclust_Nov2020_D3.0_set009.csv
└── 50
    ├── 0.01
    ├── 0.1
    └── 1.0
        └── traces_brightclust_Nov2020_D50_set007.csv

40 directories, 12 files

--------------------
../saves/firstartifact_Nov2020_train_max1set
├── 0.069
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.069_set002.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.069_set009.csv
├── 0.08
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.08_set007.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.08_set002.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.08_set004.csv
├── 0.1
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.1_set004.csv
│   ├── 0.1
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.1_set003.csv
├── 0.2
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.2_set003.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.2_set001.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.2_set009.csv
├── 0.4
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.4_set004.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.4_set002.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.4_set006.csv
├── 0.6
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.6_set010.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.6_set004.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.6_set001.csv
├── 10
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D10_set003.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D10_set006.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D10_set010.csv
├── 1.0
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D1.0_set010.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D1.0_set004.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D1.0_set001.csv
├── 3.0
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D3.0_set005.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D3.0_set010.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D3.0_set001.csv
└── 50
    ├── 0.01
    │   └── traces_brightclust_Nov2020_D50_set006.csv
    ├── 0.1
    │   └── traces_brightclust_Nov2020_D50_set009.csv
    └── 1.0
        └── traces_brightclust_Nov2020_D50_set004.csv

39 directories, 28 files

--------------------
../saves/firstartifact_Nov2020_test
├── 0.069
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.069_set005.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.069_set001.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.069_set010.csv
├── 0.08
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.08_set005.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.08_set003.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.08_set001.csv
├── 0.1
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.1_set002.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.1_set005.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.1_set001.csv
├── 0.2
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.2_set002.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.2_set007.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.2_set005.csv
├── 0.4
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.4_set008.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.4_set001.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.4_set005.csv
├── 0.6
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D0.6_set008.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D0.6_set003.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D0.6_set009.csv
├── 10
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D10_set002.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D10_set001.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D10_set005.csv
├── 1.0
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D1.0_set006.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D1.0_set003.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D1.0_set005.csv
├── 3.0
│   ├── 0.01
│   │   └── traces_brightclust_Nov2020_D3.0_set004.csv
│   ├── 0.1
│   │   └── traces_brightclust_Nov2020_D3.0_set007.csv
│   └── 1.0
│       └── traces_brightclust_Nov2020_D3.0_set002.csv
└── 50
    ├── 0.01
    │   └── traces_brightclust_Nov2020_D50_set002.csv
    ├── 0.1
    │   └── traces_brightclust_Nov2020_D50_set003.csv
    └── 1.0
        └── traces_brightclust_Nov2020_D50_set001.csv

40 directories, 30 files
2.1.2.2 Mlflow run 1 (failed mid run)
  • Use firstartifact_Nov2020_train_max2sets and firstartifact_Nov2020_val_max2sets_SORTEDIN to get a good equilibrium between amount of training data (max1set would maybe be not enough) and equal distribution of artifacts (max3sets might give too much weight to slow clusters with 0.1 speed, because a lot of them were simulated). Note that the #+RESULTS section was too long, that's why not everything is printed (but logged in MLFLOW of course) 
    mlflow run . -e search_hparams -P fluotracify_path=/beegfs/ye53nis/drmed-git/src/ -P csv_path_train=/beegfs/ye53nis/saves/firstartifact_Nov2020_train_max2sets -P csv_path_val=/beegfs/ye53nis/saves/firstartifact_Nov2020_val_max2sets_SORTEDIN -P num_session_groups=70

    0000 - val_precision0.7: 0.1635 - val_recall0.7: 0.9999 - val_tp0.9: 803718.0000 - val_fp0.9: 4111163.0000 - val_tn0.9: 180.0000 - val_fn0.9: 139.0000 - val_precision0.9: 0.1635 - val_reca
ll0.9: 0.9998 - val_accuracy: 0.1635 - val_auc: 0.5000 - val_f1: 0.2811
Epoch 6/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.5782 - tp0.1: 2622910.0000 - fp0.1: 2505997.0000 - tn0.1: 14176971.0000 - fn0.1: 354922.0000 - precision0.1: 0.5114 - r
ecall0.1: 0.8808 - tp0.3: 2342499.0000 - fp0.3: 1169993.0000 - tn0.3: 15512975.0000 - fn0.3: 635333.0000 - precision0.3: 0.6669 - recall0.3: 0.7866 - tp0.5: 2053203.0000 - fp0.5: 564864.00
00 - tn0.5: 16118104.0000 - fn0.5: 924629.0000 - precision0.5: 0.7842 - recall0.5: 0.6895 - tp0.7: 1755923.0000 - fp0.7: 242889.0000 - tn0.7: 16440079.0000 - fn0.7: 1221909.0000 - precisio
n0.7: 0.8785 - recall0.7: 0.5897 - tp0.9: 1327740.0000 - fp0.9: 66010.0000 - tn0.9: 16616958.0000 - fn0.9: 1650092.0000 - precision0.9: 0.9526 - recall0.9: 0.4459 - accuracy: 0.9242 - auc:
 0.9204 - f1: 0.2631 - val_loss: 13.4848 - val_tp0.1: 750399.0000 - val_fp0.1: 3679523.0000 - val_tn0.1: 431820.0000 - val_fn0.1: 53458.0000 - val_precision0.1: 0.1694 - val_recall0.1: 0.9
335 - val_tp0.3: 743640.0000 - val_fp0.3: 3633172.0000 - val_tn0.3: 478171.0000 - val_fn0.3: 60217.0000 - val_precision0.3: 0.1699 - val_recall0.3: 0.9251 - val_tp0.5: 739322.0000 - val_fp
0.5: 3600015.0000 - val_tn0.5: 511328.0000 - val_fn0.5: 64535.0000 - val_precision0.5: 0.1704 - val_recall0.5: 0.9197 - val_tp0.7: 734726.0000 - val_fp0.7: 3563811.0000 - val_tn0.7: 547532
.0000 - val_fn0.7: 69131.0000 - val_precision0.7: 0.1709 - val_recall0.7: 0.9140 - val_tp0.9: 727004.0000 - val_fp0.9: 3500250.0000 - val_tn0.9: 611093.0000 - val_fn0.9: 76853.0000 - val_p
recision0.9: 0.1720 - val_recall0.9: 0.9044 - val_accuracy: 0.2544 - val_auc: 0.5331 - val_f1: 0.2811
Epoch 7/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.5493 - tp0.1: 2634892.0000 - fp0.1: 2322719.0000 - tn0.1: 14360249.0000 - fn0.1: 342940.0000 - precision0.1: 0.5315 - r
ecall0.1: 0.8848 - tp0.3: 2368274.0000 - fp0.3: 1081444.0000 - tn0.3: 15601524.0000 - fn0.3: 609558.0000 - precision0.3: 0.6865 - recall0.3: 0.7953 - tp0.5: 2112193.0000 - fp0.5: 526093.00
00 - tn0.5: 16156875.0000 - fn0.5: 865639.0000 - precision0.5: 0.8006 - recall0.5: 0.7093 - tp0.7: 1831510.0000 - fp0.7: 239952.0000 - tn0.7: 16443016.0000 - fn0.7: 1146322.0000 - precisio
n0.7: 0.8842 - recall0.7: 0.6150 - tp0.9: 1405225.0000 - fp0.9: 69139.0000 - tn0.9: 16613829.0000 - fn0.9: 1572607.0000 - precision0.9: 0.9531 - recall0.9: 0.4719 - accuracy: 0.9292 - auc:
 0.9254 - f1: 0.2631 - val_loss: 7.7748 - val_tp0.1: 803044.0000 - val_fp0.1: 4110345.0000 - val_tn0.1: 998.0000 - val_fn0.1: 813.0000 - val_precision0.1: 0.1634 - val_recall0.1: 0.9990 -
val_tp0.3: 800982.0000 - val_fp0.3: 4110344.0000 - val_tn0.3: 999.0000 - val_fn0.3: 2875.0000 - val_precision0.3: 0.1631 - val_recall0.3: 0.9964 - val_tp0.5: 797547.0000 - val_fp0.5: 41103
41.0000 - val_tn0.5: 1002.0000 - val_fn0.5: 6310.0000 - val_precision0.5: 0.1625 - val_recall0.5: 0.9922 - val_tp0.7: 790114.0000 - val_fp0.7: 4110336.0000 - val_tn0.7: 1007.0000 - val_fn0
.7: 13743.0000 - val_precision0.7: 0.1612 - val_recall0.7: 0.9829 - val_tp0.9: 762283.0000 - val_fp0.9: 4110237.0000 - val_tn0.9: 1106.0000 - val_fn0.9: 41574.0000 - val_precision0.9: 0.15
64 - val_recall0.9: 0.9483 - val_accuracy: 0.1625 - val_auc: 0.4107 - val_f1: 0.2811
Epoch 8/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.5317 - tp0.1: 2659295.0000 - fp0.1: 2333208.0000 - tn0.1: 14349760.0000 - fn0.1: 318537.0000 - precision0.1: 0.5327 - r
ecall0.1: 0.8930 - tp0.3: 2408124.0000 - fp0.3: 1118900.0000 - tn0.3: 15564068.0000 - fn0.3: 569708.0000 - precision0.3: 0.6828 - recall0.3: 0.8087 - tp0.5: 2142570.0000 - fp0.5: 537358.00
00 - tn0.5: 16145610.0000 - fn0.5: 835262.0000 - precision0.5: 0.7995 - recall0.5: 0.7195 - tp0.7: 1851259.0000 - fp0.7: 238215.0000 - tn0.7: 16444753.0000 - fn0.7: 1126573.0000 - precisio
n0.7: 0.8860 - recall0.7: 0.6217 - tp0.9: 1401657.0000 - fp0.9: 63157.0000 - tn0.9: 16619811.0000 - fn0.9: 1576175.0000 - precision0.9: 0.9569 - recall0.9: 0.4707 - accuracy: 0.9302 - auc:
 0.9298 - f1: 0.2631 - val_loss: 3.0251 - val_tp0.1: 49509.0000 - val_fp0.1: 176370.0000 - val_tn0.1: 3934973.0000 - val_fn0.1: 754348.0000 - val_precision0.1: 0.2192 - val_recall0.1: 0.06
16 - val_tp0.3: 41217.0000 - val_fp0.3: 153934.0000 - val_tn0.3: 3957409.0000 - val_fn0.3: 762640.0000 - val_precision0.3: 0.2112 - val_recall0.3: 0.0513 - val_tp0.5: 36870.0000 - val_fp0.
5: 140528.0000 - val_tn0.5: 3970815.0000 - val_fn0.5: 766987.0000 - val_precision0.5: 0.2078 - val_recall0.5: 0.0459 - val_tp0.7: 32949.0000 - val_fp0.7: 127605.0000 - val_tn0.7: 3983738.0
000 - val_fn0.7: 770908.0000 - val_precision0.7: 0.2052 - val_recall0.7: 0.0410 - val_tp0.9: 27690.0000 - val_fp0.9: 108151.0000 - val_tn0.9: 4003192.0000 - val_fn0.9: 776167.0000 - val_pr
ecision0.9: 0.2038 - val_recall0.9: 0.0344 - val_accuracy: 0.8154 - val_auc: 0.5157 - val_f1: 0.2811
Epoch 9/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.5086 - tp0.1: 2674117.0000 - fp0.1: 2256419.0000 - tn0.1: 14426549.0000 - fn0.1: 303715.0000 - precision0.1: 0.5424 - r
ecall0.1: 0.8980 - tp0.3: 2438091.0000 - fp0.3: 1100372.0000 - tn0.3: 15582596.0000 - fn0.3: 539741.0000 - precision0.3: 0.6890 - recall0.3: 0.8187 - tp0.5: 2186313.0000 - fp0.5: 529438.00
00 - tn0.5: 16153530.0000 - fn0.5: 791519.0000 - precision0.5: 0.8050 - recall0.5: 0.7342 - tp0.7: 1892609.0000 - fp0.7: 231011.0000 - tn0.7: 16451957.0000 - fn0.7: 1085223.0000 - precisio
n0.7: 0.8912 - recall0.7: 0.6356 - tp0.9: 1451523.0000 - fp0.9: 61651.0000 - tn0.9: 16621317.0000 - fn0.9: 1526309.0000 - precision0.9: 0.9593 - recall0.9: 0.4874 - accuracy: 0.9328 - auc:
 0.9337 - f1: 0.2631 - val_loss: 2.5628 - val_tp0.1: 369859.0000 - val_fp0.1: 118128.0000 - val_tn0.1: 3993215.0000 - val_fn0.1: 433998.0000 - val_precision0.1: 0.7579 - val_recall0.1: 0.4
601 - val_tp0.3: 356259.0000 - val_fp0.3: 94845.0000 - val_tn0.3: 4016498.0000 - val_fn0.3: 447598.0000 - val_precision0.3: 0.7897 - val_recall0.3: 0.4432 - val_tp0.5: 347560.0000 - val_fp
0.5: 82338.0000 - val_tn0.5: 4029005.0000 - val_fn0.5: 456297.0000 - val_precision0.5: 0.8085 - val_recall0.5: 0.4324 - val_tp0.7: 338552.0000 - val_fp0.7: 70840.0000 - val_tn0.7: 4040503.
0000 - val_fn0.7: 465305.0000 - val_precision0.7: 0.8270 - val_recall0.7: 0.4212 - val_tp0.9: 317996.0000 - val_fp0.9: 52787.0000 - val_tn0.9: 4058556.0000 - val_fn0.9: 485861.0000 - val_p
recision0.9: 0.8576 - val_recall0.9: 0.3956 - val_accuracy: 0.8904 - val_auc: 0.7315 - val_f1: 0.2812
Epoch 10/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.4747 - tp0.1: 2701201.0000 - fp0.1: 2158402.0000 - tn0.1: 14524566.0000 - fn0.1: 276631.0000 - precision0.1: 0.5558 - r
ecall0.1: 0.9071 - tp0.3: 2471097.0000 - fp0.3: 1011755.0000 - tn0.3: 15671213.0000 - fn0.3: 506735.0000 - precision0.3: 0.7095 - recall0.3: 0.8298 - tp0.5: 2234639.0000 - fp0.5: 500413.00
00 - tn0.5: 16182555.0000 - fn0.5: 743193.0000 - precision0.5: 0.8170 - recall0.5: 0.7504 - tp0.7: 1960526.0000 - fp0.7: 224293.0000 - tn0.7: 16458675.0000 - fn0.7: 1017306.0000 - precisio
n0.7: 0.8973 - recall0.7: 0.6584 - tp0.9: 1515400.0000 - fp0.9: 58089.0000 - tn0.9: 16624879.0000 - fn0.9: 1462432.0000 - precision0.9: 0.9631 - recall0.9: 0.5089 - accuracy: 0.9367 - auc:
 0.9403 - f1: 0.2631 - val_loss: 2.7288 - val_tp0.1: 770447.0000 - val_fp0.1: 3861936.0000 - val_tn0.1: 249407.0000 - val_fn0.1: 33410.0000 - val_precision0.1: 0.1663 - val_recall0.1: 0.95
84 - val_tp0.3: 749545.0000 - val_fp0.3: 3747496.0000 - val_tn0.3: 363847.0000 - val_fn0.3: 54312.0000 - val_precision0.3: 0.1667 - val_recall0.3: 0.9324 - val_tp0.5: 720064.0000 - val_fp0
.5: 3582604.0000 - val_tn0.5: 528739.0000 - val_fn0.5: 83793.0000 - val_precision0.5: 0.1674 - val_recall0.5: 0.8958 - val_tp0.7: 648811.0000 - val_fp0.7: 3077696.0000 - val_tn0.7: 1033647
.0000 - val_fn0.7: 155046.0000 - val_precision0.7: 0.1741 - val_recall0.7: 0.8071 - val_tp0.9: 426993.0000 - val_fp0.9: 1072419.0000 - val_tn0.9: 3038924.0000 - val_fn0.9: 376864.0000 - va
l_precision0.9: 0.2848 - val_recall0.9: 0.5312 - val_accuracy: 0.2541 - val_auc: 0.6440 - val_f1: 0.2811
Epoch 11/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.4648 - tp0.1: 2705161.0000 - fp0.1: 2061104.0000 - tn0.1: 14621864.0000 - fn0.1: 272671.0000 - precision0.1: 0.5676 - r
ecall0.1: 0.9084 - tp0.3: 2495871.0000 - fp0.3: 1013446.0000 - tn0.3: 15669522.0000 - fn0.3: 481961.0000 - precision0.3: 0.7112 - recall0.3: 0.8382 - tp0.5: 2264736.0000 - fp0.5: 501786.00
00 - tn0.5: 16181182.0000 - fn0.5: 713096.0000 - precision0.5: 0.8186 - recall0.5: 0.7605 - tp0.7: 1995279.0000 - fp0.7: 225179.0000 - tn0.7: 16457789.0000 - fn0.7: 982553.0000 - precision
0.7: 0.8986 - recall0.7: 0.6700 - tp0.9: 1553742.0000 - fp0.9: 58202.0000 - tn0.9: 16624766.0000 - fn0.9: 1424090.0000 - precision0.9: 0.9639 - recall0.9: 0.5218 - accuracy: 0.9382 - auc:
0.9416 - f1: 0.2631 - val_loss: 2.5613 - val_tp0.1: 412390.0000 - val_fp0.1: 2079926.0000 - val_tn0.1: 2031417.0000 - val_fn0.1: 391467.0000 - val_precision0.1: 0.1655 - val_recall0.1: 0.5
130 - val_tp0.3: 404704.0000 - val_fp0.3: 2059044.0000 - val_tn0.3: 2052299.0000 - val_fn0.3: 399153.0000 - val_precision0.3: 0.1643 - val_recall0.3: 0.5035 - val_tp0.5: 396367.0000 - val_
fp0.5: 2012051.0000 - val_tn0.5: 2099292.0000 - val_fn0.5: 407490.0000 - val_precision0.5: 0.1646 - val_recall0.5: 0.4931 - val_tp0.7: 362322.0000 - val_fp0.7: 1698034.0000 - val_tn0.7: 24
13309.0000 - val_fn0.7: 441535.0000 - val_precision0.7: 0.1759 - val_recall0.7: 0.4507 - val_tp0.9: 186125.0000 - val_fp0.9: 532408.0000 - val_tn0.9: 3578935.0000 - val_fn0.9: 617732.0000
- val_precision0.9: 0.2590 - val_recall0.9: 0.2315 - val_accuracy: 0.5077 - val_auc: 0.5355 - val_f1: 0.2811
Epoch 12/20
2400/2400 [==============================] - 64s 26ms/step - loss: 0.4441 - tp0.1: 2719890.0000 - fp0.1: 2017765.0000 - tn0.1: 14665203.0000 - fn0.1: 257942.0000 - precision0.1: 0.5741 - r
ecall0.1: 0.9134 - tp0.3: 2513777.0000 - fp0.3: 998230.0000 - tn0.3: 15684738.0000 - fn0.3: 464055.0000 - precision0.3: 0.7158 - recall0.3: 0.8442 - tp0.5: 2283999.0000 - fp0.5: 506817.000
0 - tn0.5: 16176151.0000 - fn0.5: 693833.0000 - precision0.5: 0.8184 - recall0.5: 0.7670 - tp0.7: 2013621.0000 - fp0.7: 228901.0000 - tn0.7: 16454067.0000 - fn0.7: 964211.0000 - precision0
.7: 0.8979 - recall0.7: 0.6762 - tp0.9: 1553339.0000 - fp0.9: 58704.0000 - tn0.9: 16624264.0000 - fn0.9: 1424493.0000 - precision0.9: 0.9636 - recall0.9: 0.5216 - accuracy: 0.9389 - auc: 0
.9450 - f1: 0.2631 - val_loss: 7.1302 - val_tp0.1: 605571.0000 - val_fp0.1: 2024528.0000 - val_tn0.1: 2086815.0000 - val_fn0.1: 198286.0000 - val_precision0.1: 0.2302 - val_recall0.1: 0.75
33 - val_tp0.3: 600395.0000 - val_fp0.3: 1954684.0000 - val_tn0.3: 2156659.0000 - val_fn0.3: 203462.0000 - val_precision0.3: 0.2350 - val_recall0.3: 0.7469 - val_tp0.5: 597139.0000 - val_f
p0.5: 1912183.0000 - val_tn0.5: 2199160.0000 - val_fn0.5: 206718.0000 - val_precision0.5: 0.2380 - val_recall0.5: 0.7428 - val_tp0.7: 594024.0000 - val_fp0.7: 1870774.0000 - val_tn0.7: 224
0569.0000 - val_fn0.7: 209833.0000 - val_precision0.7: 0.2410 - val_recall0.7: 0.7390 - val_tp0.9: 589086.0000 - val_fp0.9: 1806611.0000 - val_tn0.9: 2304732.0000 - val_fn0.9: 214771.0000
- val_precision0.9: 0.2459 - val_recall0.9: 0.7328 - val_accuracy: 0.5689 - val_auc: 0.6516 - val_f1: 0.2811
Epoch 13/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.4185 - tp0.1: 2737060.0000 - fp0.1: 1927283.0000 - tn0.1: 14755685.0000 - fn0.1: 240772.0000 - precision0.1: 0.5868 - r
ecall0.1: 0.9191 - tp0.3: 2543087.0000 - fp0.3: 952973.0000 - tn0.3: 15729995.0000 - fn0.3: 434745.0000 - precision0.3: 0.7274 - recall0.3: 0.8540 - tp0.5: 2330175.0000 - fp0.5: 482868.000
0 - tn0.5: 16200100.0000 - fn0.5: 647657.0000 - precision0.5: 0.8283 - recall0.5: 0.7825 - tp0.7: 2076869.0000 - fp0.7: 221622.0000 - tn0.7: 16461346.0000 - fn0.7: 900963.0000 - precision0
.7: 0.9036 - recall0.7: 0.6974 - tp0.9: 1634082.0000 - fp0.9: 57693.0000 - tn0.9: 16625275.0000 - fn0.9: 1343750.0000 - precision0.9: 0.9659 - recall0.9: 0.5487 - accuracy: 0.9425 - auc: 0
.9493 - f1: 0.2631 - val_loss: 1.8139 - val_tp0.1: 239760.0000 - val_fp0.1: 4548.0000 - val_tn0.1: 4106795.0000 - val_fn0.1: 564097.0000 - val_precision0.1: 0.9814 - val_recall0.1: 0.2983
- val_tp0.3: 233628.0000 - val_fp0.3: 3848.0000 - val_tn0.3: 4107495.0000 - val_fn0.3: 570229.0000 - val_precision0.3: 0.9838 - val_recall0.3: 0.2906 - val_tp0.5: 227509.0000 - val_fp0.5:
3252.0000 - val_tn0.5: 4108091.0000 - val_fn0.5: 576348.0000 - val_precision0.5: 0.9859 - val_recall0.5: 0.2830 - val_tp0.7: 217711.0000 - val_fp0.7: 2455.0000 - val_tn0.7: 4108888.0000 -
val_fn0.7: 586146.0000 - val_precision0.7: 0.9888 - val_recall0.7: 0.2708 - val_tp0.9: 195925.0000 - val_fp0.9: 1407.0000 - val_tn0.9: 4109936.0000 - val_fn0.9: 607932.0000 - val_precision
0.9: 0.9929 - val_recall0.9: 0.2437 - val_accuracy: 0.8821 - val_auc: 0.6547 - val_f1: 0.2811
Epoch 14/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.4189 - tp0.1: 2744214.0000 - fp0.1: 1995913.0000 - tn0.1: 14687055.0000 - fn0.1: 233618.0000 - precision0.1: 0.5789 - r
ecall0.1: 0.9215 - tp0.3: 2544539.0000 - fp0.3: 941608.0000 - tn0.3: 15741360.0000 - fn0.3: 433293.0000 - precision0.3: 0.7299 - recall0.3: 0.8545 - tp0.5: 2330855.0000 - fp0.5: 475880.000
0 - tn0.5: 16207088.0000 - fn0.5: 646977.0000 - precision0.5: 0.8305 - recall0.5: 0.7827 - tp0.7: 2079586.0000 - fp0.7: 222500.0000 - tn0.7: 16460468.0000 - fn0.7: 898246.0000 - precision0
.7: 0.9033 - recall0.7: 0.6984 - tp0.9: 1628422.0000 - fp0.9: 55899.0000 - tn0.9: 16627069.0000 - fn0.9: 1349410.0000 - precision0.9: 0.9668 - recall0.9: 0.5468 - accuracy: 0.9429 - auc: 0
.9503 - f1: 0.2631 - val_loss: 1.6218 - val_tp0.1: 510355.0000 - val_fp0.1: 1689260.0000 - val_tn0.1: 2422083.0000 - val_fn0.1: 293502.0000 - val_precision0.1: 0.2320 - val_recall0.1: 0.63
49 - val_tp0.3: 421290.0000 - val_fp0.3: 1170939.0000 - val_tn0.3: 2940404.0000 - val_fn0.3: 382567.0000 - val_precision0.3: 0.2646 - val_recall0.3: 0.5241 - val_tp0.5: 358225.0000 - val_f
p0.5: 571750.0000 - val_tn0.5: 3539593.0000 - val_fn0.5: 445632.0000 - val_precision0.5: 0.3852 - val_recall0.5: 0.4456 - val_tp0.7: 323677.0000 - val_fp0.7: 214942.0000 - val_tn0.7: 38964
01.0000 - val_fn0.7: 480180.0000 - val_precision0.7: 0.6009 - val_recall0.7: 0.4027 - val_tp0.9: 290885.0000 - val_fp0.9: 143266.0000 - val_tn0.9: 3968077.0000 - val_fn0.9: 512972.0000 - v
al_precision0.9: 0.6700 - val_recall0.9: 0.3619 - val_accuracy: 0.7930 - val_auc: 0.6785 - val_f1: 0.2811
Epoch 15/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.4039 - tp0.1: 2751584.0000 - fp0.1: 1904870.0000 - tn0.1: 14778098.0000 - fn0.1: 226248.0000 - precision0.1: 0.5909 - r
ecall0.1: 0.9240 - tp0.3: 2559610.0000 - fp0.3: 920199.0000 - tn0.3: 15762769.0000 - fn0.3: 418222.0000 - precision0.3: 0.7356 - recall0.3: 0.8596 - tp0.5: 2358628.0000 - fp0.5: 480866.000
0 - tn0.5: 16202102.0000 - fn0.5: 619204.0000 - precision0.5: 0.8307 - recall0.5: 0.7921 - tp0.7: 2108242.0000 - fp0.7: 222942.0000 - tn0.7: 16460026.0000 - fn0.7: 869590.0000 - precision0
.7: 0.9044 - recall0.7: 0.7080 - tp0.9: 1658131.0000 - fp0.9: 56354.0000 - tn0.9: 16626614.0000 - fn0.9: 1319701.0000 - precision0.9: 0.9671 - recall0.9: 0.5568 - accuracy: 0.9440 - auc: 0
.9516 - f1: 0.2631 - val_loss: 1.4853 - val_tp0.1: 516528.0000 - val_fp0.1: 1043680.0000 - val_tn0.1: 3067663.0000 - val_fn0.1: 287329.0000 - val_precision0.1: 0.3311 - val_recall0.1: 0.64
26 - val_tp0.3: 418011.0000 - val_fp0.3: 456854.0000 - val_tn0.3: 3654489.0000 - val_fn0.3: 385846.0000 - val_precision0.3: 0.4778 - val_recall0.3: 0.5200 - val_tp0.5: 350573.0000 - val_fp
0.5: 253208.0000 - val_tn0.5: 3858135.0000 - val_fn0.5: 453284.0000 - val_precision0.5: 0.5806 - val_recall0.5: 0.4361 - val_tp0.7: 302626.0000 - val_fp0.7: 145263.0000 - val_tn0.7: 396608
0.0000 - val_fn0.7: 501231.0000 - val_precision0.7: 0.6757 - val_recall0.7: 0.3765 - val_tp0.9: 245404.0000 - val_fp0.9: 47550.0000 - val_tn0.9: 4063793.0000 - val_fn0.9: 558453.0000 - val
_precision0.9: 0.8377 - val_recall0.9: 0.3053 - val_accuracy: 0.8563 - val_auc: 0.7430 - val_f1: 0.2811
Epoch 16/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.3733 - tp0.1: 2769563.0000 - fp0.1: 1728895.0000 - tn0.1: 14954073.0000 - fn0.1: 208269.0000 - precision0.1: 0.6157 - r
ecall0.1: 0.9301 - tp0.3: 2599312.0000 - fp0.3: 855827.0000 - tn0.3: 15827141.0000 - fn0.3: 378520.0000 - precision0.3: 0.7523 - recall0.3: 0.8729 - tp0.5: 2413813.0000 - fp0.5: 450780.000
0 - tn0.5: 16232188.0000 - fn0.5: 564019.0000 - precision0.5: 0.8426 - recall0.5: 0.8106 - tp0.7: 2181800.0000 - fp0.7: 210158.0000 - tn0.7: 16472810.0000 - fn0.7: 796032.0000 - precision0
.7: 0.9121 - recall0.7: 0.7327 - tp0.9: 1723463.0000 - fp0.9: 51642.0000 - tn0.9: 16631326.0000 - fn0.9: 1254369.0000 - precision0.9: 0.9709 - recall0.9: 0.5788 - accuracy: 0.9484 - auc: 0
.9564 - f1: 0.2631 - val_loss: 9.3258 - val_tp0.1: 803646.0000 - val_fp0.1: 4110058.0000 - val_tn0.1: 1285.0000 - val_fn0.1: 211.0000 - val_precision0.1: 0.1636 - val_recall0.1: 0.9997 - v
al_tp0.3: 803193.0000 - val_fp0.3: 4106394.0000 - val_tn0.3: 4949.0000 - val_fn0.3: 664.0000 - val_precision0.3: 0.1636 - val_recall0.3: 0.9992 - val_tp0.5: 802499.0000 - val_fp0.5: 410051
0.0000 - val_tn0.5: 10833.0000 - val_fn0.5: 1358.0000 - val_precision0.5: 0.1637 - val_recall0.5: 0.9983 - val_tp0.7: 800907.0000 - val_fp0.7: 4088834.0000 - val_tn0.7: 22509.0000 - val_fn
0.7: 2950.0000 - val_precision0.7: 0.1638 - val_recall0.7: 0.9963 - val_tp0.9: 794980.0000 - val_fp0.9: 4040625.0000 - val_tn0.9: 70718.0000 - val_fn0.9: 8877.0000 - val_precision0.9: 0.16
44 - val_recall0.9: 0.9890 - val_accuracy: 0.1655 - val_auc: 0.5230 - val_f1: 0.2811
Epoch 17/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.3676 - tp0.1: 2769181.0000 - fp0.1: 1659194.0000 - tn0.1: 15023774.0000 - fn0.1: 208651.0000 - precision0.1: 0.6253 - r
ecall0.1: 0.9299 - tp0.3: 2610951.0000 - fp0.3: 858713.0000 - tn0.3: 15824255.0000 - fn0.3: 366881.0000 - precision0.3: 0.7525 - recall0.3: 0.8768 - tp0.5: 2429189.0000 - fp0.5: 452947.000
0 - tn0.5: 16230021.0000 - fn0.5: 548643.0000 - precision0.5: 0.8428 - recall0.5: 0.8158 - tp0.7: 2189579.0000 - fp0.7: 210955.0000 - tn0.7: 16472013.0000 - fn0.7: 788253.0000 - precision0
.7: 0.9121 - recall0.7: 0.7353 - tp0.9: 1747880.0000 - fp0.9: 50128.0000 - tn0.9: 16632840.0000 - fn0.9: 1229952.0000 - precision0.9: 0.9721 - recall0.9: 0.5870 - accuracy: 0.9491 - auc: 0
.9570 - f1: 0.2631 - val_loss: 5.5868 - val_tp0.1: 802997.0000 - val_fp0.1: 4076916.0000 - val_tn0.1: 34427.0000 - val_fn0.1: 860.0000 - val_precision0.1: 0.1646 - val_recall0.1: 0.9989 -
val_tp0.3: 802425.0000 - val_fp0.3: 4060844.0000 - val_tn0.3: 50499.0000 - val_fn0.3: 1432.0000 - val_precision0.3: 0.1650 - val_recall0.3: 0.9982 - val_tp0.5: 801861.0000 - val_fp0.5: 404
2504.0000 - val_tn0.5: 68839.0000 - val_fn0.5: 1996.0000 - val_precision0.5: 0.1655 - val_recall0.5: 0.9975 - val_tp0.7: 800923.0000 - val_fp0.7: 4011295.0000 - val_tn0.7: 100048.0000 - va
l_fn0.7: 2934.0000 - val_precision0.7: 0.1664 - val_recall0.7: 0.9964 - val_tp0.9: 795379.0000 - val_fp0.9: 3913030.0000 - val_tn0.9: 198313.0000 - val_fn0.9: 8478.0000 - val_precision0.9:
 0.1689 - val_recall0.9: 0.9895 - val_accuracy: 0.1771 - val_auc: 0.5462 - val_f1: 0.2811
Epoch 18/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.3717 - tp0.1: 2768914.0000 - fp0.1: 1725165.0000 - tn0.1: 14957803.0000 - fn0.1: 208918.0000 - precision0.1: 0.6161 - r
ecall0.1: 0.9298 - tp0.3: 2598193.0000 - fp0.3: 861638.0000 - tn0.3: 15821330.0000 - fn0.3: 379639.0000 - precision0.3: 0.7510 - recall0.3: 0.8725 - tp0.5: 2411640.0000 - fp0.5: 445576.000
0 - tn0.5: 16237392.0000 - fn0.5: 566192.0000 - precision0.5: 0.8441 - recall0.5: 0.8099 - tp0.7: 2177668.0000 - fp0.7: 207154.0000 - tn0.7: 16475814.0000 - fn0.7: 800164.0000 - precision0
.7: 0.9131 - recall0.7: 0.7313 - tp0.9: 1736637.0000 - fp0.9: 49527.0000 - tn0.9: 16633441.0000 - fn0.9: 1241195.0000 - precision0.9: 0.9723 - recall0.9: 0.5832 - accuracy: 0.9485 - auc: 0
.9563 - f1: 0.2631 - val_loss: 1.5014 - val_tp0.1: 559528.0000 - val_fp0.1: 1842714.0000 - val_tn0.1: 2268629.0000 - val_fn0.1: 244329.0000 - val_precision0.1: 0.2329 - val_recall0.1: 0.69
61 - val_tp0.3: 502697.0000 - val_fp0.3: 1389421.0000 - val_tn0.3: 2721922.0000 - val_fn0.3: 301160.0000 - val_precision0.3: 0.2657 - val_recall0.3: 0.6254 - val_tp0.5: 413701.0000 - val_f
p0.5: 665286.0000 - val_tn0.5: 3446057.0000 - val_fn0.5: 390156.0000 - val_precision0.5: 0.3834 - val_recall0.5: 0.5146 - val_tp0.7: 345100.0000 - val_fp0.7: 322127.0000 - val_tn0.7: 37892
16.0000 - val_fn0.7: 458757.0000 - val_precision0.7: 0.5172 - val_recall0.7: 0.4293 - val_tp0.9: 274540.0000 - val_fp0.9: 106793.0000 - val_tn0.9: 4004550.0000 - val_fn0.9: 529317.0000 - v
al_precision0.9: 0.7199 - val_recall0.9: 0.3415 - val_accuracy: 0.7853 - val_auc: 0.7127 - val_f1: 0.2811
Epoch 19/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.3547 - tp0.1: 2782451.0000 - fp0.1: 1665578.0000 - tn0.1: 15017390.0000 - fn0.1: 195381.0000 - precision0.1: 0.6255 - r
ecall0.1: 0.9344 - tp0.3: 2619177.0000 - fp0.3: 850031.0000 - tn0.3: 15832937.0000 - fn0.3: 358655.0000 - precision0.3: 0.7550 - recall0.3: 0.8796 - tp0.5: 2432227.0000 - fp0.5: 440386.000
0 - tn0.5: 16242582.0000 - fn0.5: 545605.0000 - precision0.5: 0.8467 - recall0.5: 0.8168 - tp0.7: 2203211.0000 - fp0.7: 207630.0000 - tn0.7: 16475338.0000 - fn0.7: 774621.0000 - precision0
.7: 0.9139 - recall0.7: 0.7399 - tp0.9: 1758972.0000 - fp0.9: 48353.0000 - tn0.9: 16634615.0000 - fn0.9: 1218860.0000 - precision0.9: 0.9732 - recall0.9: 0.5907 - accuracy: 0.9499 - auc: 0
.9594 - f1: 0.2631 - val_loss: 1.6122 - val_tp0.1: 364867.0000 - val_fp0.1: 80402.0000 - val_tn0.1: 4030941.0000 - val_fn0.1: 438990.0000 - val_precision0.1: 0.8194 - val_recall0.1: 0.4539
 - val_tp0.3: 338563.0000 - val_fp0.3: 52446.0000 - val_tn0.3: 4058897.0000 - val_fn0.3: 465294.0000 - val_precision0.3: 0.8659 - val_recall0.3: 0.4212 - val_tp0.5: 303084.0000 - val_fp0.5
: 26254.0000 - val_tn0.5: 4085089.0000 - val_fn0.5: 500773.0000 - val_precision0.5: 0.9203 - val_recall0.5: 0.3770 - val_tp0.7: 268991.0000 - val_fp0.7: 12371.0000 - val_tn0.7: 4098972.000
0 - val_fn0.7: 534866.0000 - val_precision0.7: 0.9560 - val_recall0.7: 0.3346 - val_tp0.9: 228930.0000 - val_fp0.9: 4729.0000 - val_tn0.9: 4106614.0000 - val_fn0.9: 574927.0000 - val_preci
sion0.9: 0.9798 - val_recall0.9: 0.2848 - val_accuracy: 0.8928 - val_auc: 0.7252 - val_f1: 0.2811
Epoch 20/20
2400/2400 [==============================] - 63s 26ms/step - loss: 0.3465 - tp0.1: 2789105.0000 - fp0.1: 1630391.0000 - tn0.1: 15052577.0000 - fn0.1: 188727.0000 - precision0.1: 0.6311 - r
ecall0.1: 0.9366 - tp0.3: 2634590.0000 - fp0.3: 826424.0000 - tn0.3: 15856544.0000 - fn0.3: 343242.0000 - precision0.3: 0.7612 - recall0.3: 0.8847 - tp0.5: 2453024.0000 - fp0.5: 430471.000
0 - tn0.5: 16252497.0000 - fn0.5: 524808.0000 - precision0.5: 0.8507 - recall0.5: 0.8238 - tp0.7: 2228295.0000 - fp0.7: 203336.0000 - tn0.7: 16479632.0000 - fn0.7: 749537.0000 - precision0
.7: 0.9164 - recall0.7: 0.7483 - tp0.9: 1795290.0000 - fp0.9: 47888.0000 - tn0.9: 16635080.0000 - fn0.9: 1182542.0000 - precision0.9: 0.9740 - recall0.9: 0.6029 - accuracy: 0.9514 - auc: 0
.9610 - f1: 0.2631 - val_loss: 1.6856 - val_tp0.1: 306188.0000 - val_fp0.1: 33472.0000 - val_tn0.1: 4077871.0000 - val_fn0.1: 497669.0000 - val_precision0.1: 0.9015 - val_recall0.1: 0.3809
 - val_tp0.3: 252310.0000 - val_fp0.3: 16537.0000 - val_tn0.3: 4094806.0000 - val_fn0.3: 551547.0000 - val_precision0.3: 0.9385 - val_recall0.3: 0.3139 - val_tp0.5: 216053.0000 - val_fp0.5
: 9308.0000 - val_tn0.5: 4102035.0000 - val_fn0.5: 587804.0000 - val_precision0.5: 0.9587 - val_recall0.5: 0.2688 - val_tp0.7: 183648.0000 - val_fp0.7: 4739.0000 - val_tn0.7: 4106604.0000
- val_fn0.7: 620209.0000 - val_precision0.7: 0.9748 - val_recall0.7: 0.2285 - val_tp0.9: 141556.0000 - val_fp0.9: 1421.0000 - val_tn0.9: 4109922.0000 - val_fn0.9: 662301.0000 - val_precisi
on0.9: 0.9901 - val_recall0.9: 0.1761 - val_accuracy: 0.8785 - val_auc: 0.7019 - val_f1: 0.2811
--- Running training session 55/140
{'hp_epochs': 20, 'hp_batch_size': 10, 'hp_scaler': 'robust', 'hp_n_levels': 5, 'hp_first_filters': 16, 'hp_pool_size': 4, 'hp_input_size': 8192, 'hp_lr_start': 0.06276763366515732, 'hp_lr
_power': 1.0}
--- repeat #: 1
input - shape:   (None, 8192, 1)
output - shape:  (None, 8192, 1)
Epoch 1/20
480/480 [==============================] - 45s 53ms/step - loss: 0.7092 - tp0.1: 5321969.0000 - fp0.1: 6687422.0000 - tn0.1: 26697388.0000 - fn0.1: 614807.0000 - precision0.1: 0.4432 - rec
all0.1: 0.8964 - tp0.3: 4646941.0000 - fp0.3: 2819314.0000 - tn0.3: 30565520.0000 - fn0.3: 1289835.0000 - precision0.3: 0.6224 - recall0.3: 0.7827 - tp0.5: 3895656.0000 - fp0.5: 1079912.00
00 - tn0.5: 32304924.0000 - fn0.5: 2041120.0000 - precision0.5: 0.7830 - recall0.5: 0.6562 - tp0.7: 3225404.0000 - fp0.7: 417241.0000 - tn0.7: 32967584.0000 - fn0.7: 2711372.0000 - precisi
on0.7: 0.8855 - recall0.7: 0.5433 - tp0.9: 2363272.0000 - fp0.9: 80854.0000 - tn0.9: 33303980.0000 - fn0.9: 3573504.0000 - precision0.9: 0.9669 - recall0.9: 0.3981 - accuracy: 0.9206 - auc
: 0.9209 - f1: 0.2624 - val_loss: 1.7392 - val_tp0.1: 675173.0000 - val_fp0.1: 51103.0000 - val_tn0.1: 8196692.0000 - val_fn0.1: 907432.0000 - val_precision0.1: 0.9296 - val_recall0.1: 0.4
266 - val_tp0.3: 629304.0000 - val_fp0.3: 27265.0000 - val_tn0.3: 8220530.0000 - val_fn0.3: 953301.0000 - val_precision0.3: 0.9585 - val_recall0.3: 0.3976 - val_tp0.5: 554968.0000 - val_fp
0.5: 8081.0000 - val_tn0.5: 8239714.0000 - val_fn0.5: 1027637.0000 - val_precision0.5: 0.9856 - val_recall0.5: 0.3507 - val_tp0.7: 482396.0000 - val_fp0.7: 1352.0000 - val_tn0.7: 8246443.0
000 - val_fn0.7: 1100209.0000 - val_precision0.7: 0.9972 - val_recall0.7: 0.3048 - val_tp0.9: 365703.0000 - val_fp0.9: 93.0000 - val_tn0.9: 8247702.0000 - val_fn0.9: 1216902.0000 - val_pre
cision0.9: 0.9997 - val_recall0.9: 0.2311 - val_accuracy: 0.8946 - val_auc: 0.7140 - val_f1: 0.2773
Epoch 2/20
480/480 [==============================] - 21s 44ms/step - loss: 0.5057 - tp0.1: 5497302.0000 - fp0.1: 6092525.0000 - tn0.1: 27292284.0000 - fn0.1: 439474.0000 - precision0.1: 0.4743 - rec
all0.1: 0.9260 - tp0.3: 4832858.0000 - fp0.3: 2368876.0000 - tn0.3: 31015944.0000 - fn0.3: 1103918.0000 - precision0.3: 0.6711 - recall0.3: 0.8141 - tp0.5: 4194498.0000 - fp0.5: 1025948.00
00 - tn0.5: 32358848.0000 - fn0.5: 1742278.0000 - precision0.5: 0.8035 - recall0.5: 0.7065 - tp0.7: 3542699.0000 - fp0.7: 455958.0000 - tn0.7: 32928884.0000 - fn0.7: 2394077.0000 - precisi
on0.7: 0.8860 - recall0.7: 0.5967 - tp0.9: 2649175.0000 - fp0.9: 121325.0000 - tn0.9: 33263512.0000 - fn0.9: 3287601.0000 - precision0.9: 0.9562 - recall0.9: 0.4462 - accuracy: 0.9296 - au
c: 0.9389 - f1: 0.2624 - val_loss: 1.7645 - val_tp0.1: 734127.0000 - val_fp0.1: 22185.0000 - val_tn0.1: 8225610.0000 - val_fn0.1: 848478.0000 - val_precision0.1: 0.9707 - val_recall0.1: 0.
4639 - val_tp0.3: 692477.0000 - val_fp0.3: 8041.0000 - val_tn0.3: 8239754.0000 - val_fn0.3: 890128.0000 - val_precision0.3: 0.9885 - val_recall0.3: 0.4376 - val_tp0.5: 645446.0000 - val_fp
0.5: 3920.0000 - val_tn0.5: 8243875.0000 - val_fn0.5: 937159.0000 - val_precision0.5: 0.9940 - val_recall0.5: 0.4078 - val_tp0.7: 574282.0000 - val_fp0.7: 1352.0000 - val_tn0.7: 8246443.00
00 - val_fn0.7: 1008323.0000 - val_precision0.7: 0.9977 - val_recall0.7: 0.3629 - val_tp0.9: 453034.0000 - val_fp0.9: 218.0000 - val_tn0.9: 8247577.0000 - val_fn0.9: 1129571.0000 - val_pre
cision0.9: 0.9995 - val_recall0.9: 0.2863 - val_accuracy: 0.9043 - val_auc: 0.7331 - val_f1: 0.2773
Epoch 3/20
480/480 [==============================] - 21s 44ms/step - loss: 0.4505 - tp0.1: 5568258.0000 - fp0.1: 5463889.0000 - tn0.1: 27920928.0000 - fn0.1: 368518.0000 - precision0.1: 0.5047 - rec
all0.1: 0.9379 - tp0.3: 4929910.0000 - fp0.3: 2020777.0000 - tn0.3: 31364036.0000 - fn0.3: 1006866.0000 - precision0.3: 0.7093 - recall0.3: 0.8304 - tp0.5: 4449095.0000 - fp0.5: 1041240.00
00 - tn0.5: 32343572.0000 - fn0.5: 1487681.0000 - precision0.5: 0.8104 - recall0.5: 0.7494 - tp0.7: 3823844.0000 - fp0.7: 479840.0000 - tn0.7: 32904986.0000 - fn0.7: 2112932.0000 - precisi
on0.7: 0.8885 - recall0.7: 0.6441 - tp0.9: 2882785.0000 - fp0.9: 126351.0000 - tn0.9: 33258496.0000 - fn0.9: 3053991.0000 - precision0.9: 0.9580 - recall0.9: 0.4856 - accuracy: 0.9357 - au
c: 0.9475 - f1: 0.2624 - val_loss: 0.6952 - val_tp0.1: 1307421.0000 - val_fp0.1: 509799.0000 - val_tn0.1: 7737996.0000 - val_fn0.1: 275184.0000 - val_precision0.1: 0.7195 - val_recall0.1:
0.8261 - val_tp0.3: 1160591.0000 - val_fp0.3: 197075.0000 - val_tn0.3: 8050720.0000 - val_fn0.3: 422014.0000 - val_precision0.3: 0.8548 - val_recall0.3: 0.7333 - val_tp0.5: 946439.0000 - v
al_fp0.5: 19008.0000 - val_tn0.5: 8228787.0000 - val_fn0.5: 636166.0000 - val_precision0.5: 0.9803 - val_recall0.5: 0.5980 - val_tp0.7: 695971.0000 - val_fp0.7: 2696.0000 - val_tn0.7: 8245
099.0000 - val_fn0.7: 886634.0000 - val_precision0.7: 0.9961 - val_recall0.7: 0.4398 - val_tp0.9: 409103.0000 - val_fp0.9: 135.0000 - val_tn0.9: 8247660.0000 - val_fn0.9: 1173502.0000 - va
l_precision0.9: 0.9997 - val_recall0.9: 0.2585 - val_accuracy: 0.9334 - val_auc: 0.9043 - val_f1: 0.2773
Epoch 4/20
480/480 [==============================] - 21s 44ms/step - loss: 0.4147 - tp0.1: 5582837.0000 - fp0.1: 4800957.0000 - tn0.1: 28583876.0000 - fn0.1: 353939.0000 - precision0.1: 0.5376 - rec
all0.1: 0.9404 - tp0.3: 5025077.0000 - fp0.3: 1931724.0000 - tn0.3: 31453094.0000 - fn0.3: 911699.0000 - precision0.3: 0.7223 - recall0.3: 0.8464 - tp0.5: 4590126.0000 - fp0.5: 953938.0000
 - tn0.5: 32430896.0000 - fn0.5: 1346650.0000 - precision0.5: 0.8279 - recall0.5: 0.7732 - tp0.7: 4038584.0000 - fp0.7: 465004.0000 - tn0.7: 32919816.0000 - fn0.7: 1898192.0000 - precision
0.7: 0.8967 - recall0.7: 0.6803 - tp0.9: 3025476.0000 - fp0.9: 119273.0000 - tn0.9: 33265544.0000 - fn0.9: 2911300.0000 - precision0.9: 0.9621 - recall0.9: 0.5096 - accuracy: 0.9415 - auc:
 0.9527 - f1: 0.2624 - val_loss: 0.5201 - val_tp0.1: 1399126.0000 - val_fp0.1: 622073.0000 - val_tn0.1: 7625722.0000 - val_fn0.1: 183479.0000 - val_precision0.1: 0.6922 - val_recall0.1: 0.
8841 - val_tp0.3: 1113548.0000 - val_fp0.3: 36502.0000 - val_tn0.3: 8211293.0000 - val_fn0.3: 469057.0000 - val_precision0.3: 0.9683 - val_recall0.3: 0.7036 - val_tp0.5: 884202.0000 - val_
fp0.5: 4823.0000 - val_tn0.5: 8242972.0000 - val_fn0.5: 698403.0000 - val_precision0.5: 0.9946 - val_recall0.5: 0.5587 - val_tp0.7: 569297.0000 - val_fp0.7: 628.0000 - val_tn0.7: 8247167.0
000 - val_fn0.7: 1013308.0000 - val_precision0.7: 0.9989 - val_recall0.7: 0.3597 - val_tp0.9: 374108.0000 - val_fp0.9: 50.0000 - val_tn0.9: 8247745.0000 - val_fn0.9: 1208497.0000 - val_pre
cision0.9: 0.9999 - val_recall0.9: 0.2364 - val_accuracy: 0.9285 - val_auc: 0.9333 - val_f1: 0.2773
Epoch 5/20
480/480 [==============================] - 21s 44ms/step - loss: 0.3754 - tp0.1: 5634883.0000 - fp0.1: 4427128.0000 - tn0.1: 28957694.0000 - fn0.1: 301893.0000 - precision0.1: 0.5600 - rec
all0.1: 0.9491 - tp0.3: 5093958.0000 - fp0.3: 1784747.0000 - tn0.3: 31600080.0000 - fn0.3: 842818.0000 - precision0.3: 0.7405 - recall0.3: 0.8580 - tp0.5: 4689585.0000 - fp0.5: 889979.0000
 - tn0.5: 32494848.0000 - fn0.5: 1247191.0000 - precision0.5: 0.8405 - recall0.5: 0.7899 - tp0.7: 4192223.0000 - fp0.7: 451284.0000 - tn0.7: 32933542.0000 - fn0.7: 1744553.0000 - precision
0.7: 0.9028 - recall0.7: 0.7061 - tp0.9: 3191877.0000 - fp0.9: 120760.0000 - tn0.9: 33264048.0000 - fn0.9: 2744899.0000 - precision0.9: 0.9635 - recall0.9: 0.5376 - accuracy: 0.9456 - auc:
 0.9584 - f1: 0.2624 - val_loss: 0.4916 - val_tp0.1: 1492248.0000 - val_fp0.1: 1196998.0000 - val_tn0.1: 7050797.0000 - val_fn0.1: 90357.0000 - val_precision0.1: 0.5549 - val_recall0.1: 0.
9429 - val_tp0.3: 948440.0000 - val_fp0.3: 76049.0000 - val_tn0.3: 8171746.0000 - val_fn0.3: 634165.0000 - val_precision0.3: 0.9258 - val_recall0.3: 0.5993 - val_tp0.5: 790703.0000 - val_f
p0.5: 18060.0000 - val_tn0.5: 8229735.0000 - val_fn0.5: 791902.0000 - val_precision0.5: 0.9777 - val_recall0.5: 0.4996 - val_tp0.7: 644735.0000 - val_fp0.7: 3337.0000 - val_tn0.7: 8244458.
0000 - val_fn0.7: 937870.0000 - val_precision0.7: 0.9949 - val_recall0.7: 0.4074 - val_tp0.9: 446636.0000 - val_fp0.9: 194.0000 - val_tn0.9: 8247601.0000 - val_fn0.9: 1135969.0000 - val_pr
ecision0.9: 0.9996 - val_recall0.9: 0.2822 - val_accuracy: 0.9176 - val_auc: 0.9509 - val_f1: 0.2773
Epoch 6/20
480/480 [==============================] - 21s 44ms/step - loss: 0.3576 - tp0.1: 5650782.0000 - fp0.1: 4214930.0000 - tn0.1: 29169904.0000 - fn0.1: 285994.0000 - precision0.1: 0.5728 - rec
all0.1: 0.9518 - tp0.3: 5157731.0000 - fp0.3: 1802746.0000 - tn0.3: 31582076.0000 - fn0.3: 779045.0000 - precision0.3: 0.7410 - recall0.3: 0.8688 - tp0.5: 4706560.0000 - fp0.5: 815239.0000
 - tn0.5: 32569602.0000 - fn0.5: 1230216.0000 - precision0.5: 0.8524 - recall0.5: 0.7928 - tp0.7: 4257269.0000 - fp0.7: 416618.0000 - tn0.7: 32968196.0000 - fn0.7: 1679507.0000 - precision
0.7: 0.9109 - recall0.7: 0.7171 - tp0.9: 3338555.0000 - fp0.9: 117993.0000 - tn0.9: 33266826.0000 - fn0.9: 2598221.0000 - precision0.9: 0.9659 - recall0.9: 0.5624 - accuracy: 0.9480 - auc:
 0.9615 - f1: 0.2624 - val_loss: 0.5022 - val_tp0.1: 1476921.0000 - val_fp0.1: 1294149.0000 - val_tn0.1: 6953646.0000 - val_fn0.1: 105684.0000 - val_precision0.1: 0.5330 - val_recall0.1: 0
.9332 - val_tp0.3: 1409625.0000 - val_fp0.3: 897663.0000 - val_tn0.3: 7350132.0000 - val_fn0.3: 172980.0000 - val_precision0.3: 0.6109 - val_recall0.3: 0.8907 - val_tp0.5: 1288431.0000 - v
al_fp0.5: 516719.0000 - val_tn0.5: 7731076.0000 - val_fn0.5: 294174.0000 - val_precision0.5: 0.7138 - val_recall0.5: 0.8141 - val_tp0.7: 1184168.0000 - val_fp0.7: 362179.0000 - val_tn0.7:
7885616.0000 - val_fn0.7: 398437.0000 - val_precision0.7: 0.7658 - val_recall0.7: 0.7482 - val_tp0.9: 982390.0000 - val_fp0.9: 193676.0000 - val_tn0.9: 8054119.0000 - val_fn0.9: 600215.000
0 - val_precision0.9: 0.8353 - val_recall0.9: 0.6207 - val_accuracy: 0.9175 - val_auc: 0.9394 - val_f1: 0.2773
Epoch 7/20
480/480 [==============================] - 21s 44ms/step - loss: 0.3384 - tp0.1: 5645527.0000 - fp0.1: 3726823.0000 - tn0.1: 29657992.0000 - fn0.1: 291249.0000 - precision0.1: 0.6024 - rec
all0.1: 0.9509 - tp0.3: 5236539.0000 - fp0.3: 1794315.0000 - tn0.3: 31590506.0000 - fn0.3: 700237.0000 - precision0.3: 0.7448 - recall0.3: 0.8821 - tp0.5: 4787739.0000 - fp0.5: 779171.0000
 - tn0.5: 32605672.0000 - fn0.5: 1149037.0000 - precision0.5: 0.8600 - recall0.5: 0.8065 - tp0.7: 4390618.0000 - fp0.7: 404832.0000 - tn0.7: 32980006.0000 - fn0.7: 1546158.0000 - precision
0.7: 0.9156 - recall0.7: 0.7396 - tp0.9: 3468567.0000 - fp0.9: 102570.0000 - tn0.9: 33282264.0000 - fn0.9: 2468209.0000 - precision0.9: 0.9713 - recall0.9: 0.5843 - accuracy: 0.9510 - auc:
 0.9638 - f1: 0.2624 - val_loss: 0.3821 - val_tp0.1: 1458280.0000 - val_fp0.1: 507037.0000 - val_tn0.1: 7740758.0000 - val_fn0.1: 124325.0000 - val_precision0.1: 0.7420 - val_recall0.1: 0.
9214 - val_tp0.3: 1138703.0000 - val_fp0.3: 109160.0000 - val_tn0.3: 8138635.0000 - val_fn0.3: 443902.0000 - val_precision0.3: 0.9125 - val_recall0.3: 0.7195 - val_tp0.5: 963399.0000 - val
_fp0.5: 44369.0000 - val_tn0.5: 8203426.0000 - val_fn0.5: 619206.0000 - val_precision0.5: 0.9560 - val_recall0.5: 0.6087 - val_tp0.7: 768098.0000 - val_fp0.7: 13185.0000 - val_tn0.7: 82346
10.0000 - val_fn0.7: 814507.0000 - val_precision0.7: 0.9831 - val_recall0.7: 0.4853 - val_tp0.9: 549365.0000 - val_fp0.9: 1330.0000 - val_tn0.9: 8246465.0000 - val_fn0.9: 1033240.0000 - va
l_precision0.9: 0.9976 - val_recall0.9: 0.3471 - val_accuracy: 0.9325 - val_auc: 0.9604 - val_f1: 0.2773
Epoch 8/20
480/480 [==============================] - 21s 44ms/step - loss: 0.3065 - tp0.1: 5676575.0000 - fp0.1: 3449099.0000 - tn0.1: 29935738.0000 - fn0.1: 260201.0000 - precision0.1: 0.6220 - rec
all0.1: 0.9562 - tp0.3: 5325689.0000 - fp0.3: 1652021.0000 - tn0.3: 31732808.0000 - fn0.3: 611087.0000 - precision0.3: 0.7632 - recall0.3: 0.8971 - tp0.5: 4859260.0000 - fp0.5: 655135.0000
 - tn0.5: 32729690.0000 - fn0.5: 1077516.0000 - precision0.5: 0.8812 - recall0.5: 0.8185 - tp0.7: 4528001.0000 - fp0.7: 355303.0000 - tn0.7: 33029518.0000 - fn0.7: 1408775.0000 - precision
0.7: 0.9272 - recall0.7: 0.7627 - tp0.9: 3797501.0000 - fp0.9: 104696.0000 - tn0.9: 33280128.0000 - fn0.9: 2139275.0000 - precision0.9: 0.9732 - recall0.9: 0.6397 - accuracy: 0.9559 - auc:
 0.9683 - f1: 0.2624 - val_loss: 0.4780 - val_tp0.1: 1324114.0000 - val_fp0.1: 275210.0000 - val_tn0.1: 7972585.0000 - val_fn0.1: 258491.0000 - val_precision0.1: 0.8279 - val_recall0.1: 0.
8367 - val_tp0.3: 1141828.0000 - val_fp0.3: 54316.0000 - val_tn0.3: 8193479.0000 - val_fn0.3: 440777.0000 - val_precision0.3: 0.9546 - val_recall0.3: 0.7215 - val_tp0.5: 1048254.0000 - val
_fp0.5: 27301.0000 - val_tn0.5: 8220494.0000 - val_fn0.5: 534351.0000 - val_precision0.5: 0.9746 - val_recall0.5: 0.6624 - val_tp0.7: 938136.0000 - val_fp0.7: 13368.0000 - val_tn0.7: 82344
27.0000 - val_fn0.7: 644469.0000 - val_precision0.7: 0.9860 - val_recall0.7: 0.5928 - val_tp0.9: 734879.0000 - val_fp0.9: 3835.0000 - val_tn0.9: 8243960.0000 - val_fn0.9: 847726.0000 - val
_precision0.9: 0.9948 - val_recall0.9: 0.4643 - val_accuracy: 0.9429 - val_auc: 0.9295 - val_f1: 0.2773
Epoch 9/20
480/480 [==============================] - 21s 44ms/step - loss: 0.3010 - tp0.1: 5676976.0000 - fp0.1: 3336526.0000 - tn0.1: 30048290.0000 - fn0.1: 259800.0000 - precision0.1: 0.6298 - rec
all0.1: 0.9562 - tp0.3: 5326144.0000 - fp0.3: 1579793.0000 - tn0.3: 31805038.0000 - fn0.3: 610632.0000 - precision0.3: 0.7712 - recall0.3: 0.8971 - tp0.5: 4898244.0000 - fp0.5: 689875.0000
 - tn0.5: 32694952.0000 - fn0.5: 1038532.0000 - precision0.5: 0.8765 - recall0.5: 0.8251 - tp0.7: 4562202.0000 - fp0.7: 365675.0000 - tn0.7: 33019140.0000 - fn0.7: 1374574.0000 - precision
0.7: 0.9258 - recall0.7: 0.7685 - tp0.9: 3784831.0000 - fp0.9: 104828.0000 - tn0.9: 33280008.0000 - fn0.9: 2151945.0000 - precision0.9: 0.9730 - recall0.9: 0.6375 - accuracy: 0.9560 - auc:
 0.9694 - f1: 0.2624 - val_loss: 0.4138 - val_tp0.1: 1542379.0000 - val_fp0.1: 1309329.0000 - val_tn0.1: 6938466.0000 - val_fn0.1: 40226.0000 - val_precision0.1: 0.5409 - val_recall0.1: 0.
9746 - val_tp0.3: 1522585.0000 - val_fp0.3: 940842.0000 - val_tn0.3: 7306953.0000 - val_fn0.3: 60020.0000 - val_precision0.3: 0.6181 - val_recall0.3: 0.9621 - val_tp0.5: 1471806.0000 - val
_fp0.5: 673062.0000 - val_tn0.5: 7574733.0000 - val_fn0.5: 110799.0000 - val_precision0.5: 0.6862 - val_recall0.5: 0.9300 - val_tp0.7: 1439073.0000 - val_fp0.7: 504700.0000 - val_tn0.7: 77
43095.0000 - val_fn0.7: 143532.0000 - val_precision0.7: 0.7404 - val_recall0.7: 0.9093 - val_tp0.9: 1349827.0000 - val_fp0.9: 275289.0000 - val_tn0.9: 7972506.0000 - val_fn0.9: 232778.0000
 - val_precision0.9: 0.8306 - val_recall0.9: 0.8529 - val_accuracy: 0.9203 - val_auc: 0.9720 - val_f1: 0.2773
Epoch 10/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2783 - tp0.1: 5687717.0000 - fp0.1: 2979751.0000 - tn0.1: 30405060.0000 - fn0.1: 249059.0000 - precision0.1: 0.6562 - rec
all0.1: 0.9580 - tp0.3: 5401647.0000 - fp0.3: 1520293.0000 - tn0.3: 31864510.0000 - fn0.3: 535129.0000 - precision0.3: 0.7804 - recall0.3: 0.9099 - tp0.5: 4984841.0000 - fp0.5: 648600.0000
 - tn0.5: 32736208.0000 - fn0.5: 951935.0000 - precision0.5: 0.8849 - recall0.5: 0.8397 - tp0.7: 4668237.0000 - fp0.7: 338935.0000 - tn0.7: 33045886.0000 - fn0.7: 1268539.0000 - precision0
.7: 0.9323 - recall0.7: 0.7863 - tp0.9: 3974497.0000 - fp0.9: 95618.0000 - tn0.9: 33289212.0000 - fn0.9: 1962279.0000 - precision0.9: 0.9765 - recall0.9: 0.6695 - accuracy: 0.9593 - auc: 0
.9717 - f1: 0.2624 - val_loss: 0.3033 - val_tp0.1: 1528072.0000 - val_fp0.1: 891324.0000 - val_tn0.1: 7356471.0000 - val_fn0.1: 54533.0000 - val_precision0.1: 0.6316 - val_recall0.1: 0.965
5 - val_tp0.3: 1410892.0000 - val_fp0.3: 321988.0000 - val_tn0.3: 7925807.0000 - val_fn0.3: 171713.0000 - val_precision0.3: 0.8142 - val_recall0.3: 0.8915 - val_tp0.5: 1288259.0000 - val_f
p0.5: 124605.0000 - val_tn0.5: 8123190.0000 - val_fn0.5: 294346.0000 - val_precision0.5: 0.9118 - val_recall0.5: 0.8140 - val_tp0.7: 1162405.0000 - val_fp0.7: 53667.0000 - val_tn0.7: 81941
28.0000 - val_fn0.7: 420200.0000 - val_precision0.7: 0.9559 - val_recall0.7: 0.7345 - val_tp0.9: 912340.0000 - val_fp0.9: 11304.0000 - val_tn0.9: 8236491.0000 - val_fn0.9: 670265.0000 - va
l_precision0.9: 0.9878 - val_recall0.9: 0.5765 - val_accuracy: 0.9574 - val_auc: 0.9736 - val_f1: 0.2773
Epoch 11/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2692 - tp0.1: 5698307.0000 - fp0.1: 2926299.0000 - tn0.1: 30458524.0000 - fn0.1: 238469.0000 - precision0.1: 0.6607 - rec
all0.1: 0.9598 - tp0.3: 5414811.0000 - fp0.3: 1480385.0000 - tn0.3: 31904448.0000 - fn0.3: 521965.0000 - precision0.3: 0.7853 - recall0.3: 0.9121 - tp0.5: 4973314.0000 - fp0.5: 587269.0000
 - tn0.5: 32797544.0000 - fn0.5: 963462.0000 - precision0.5: 0.8944 - recall0.5: 0.8377 - tp0.7: 4687698.0000 - fp0.7: 318968.0000 - tn0.7: 33065868.0000 - fn0.7: 1249078.0000 - precision0
.7: 0.9363 - recall0.7: 0.7896 - tp0.9: 4069138.0000 - fp0.9: 90368.0000 - tn0.9: 33294464.0000 - fn0.9: 1867638.0000 - precision0.9: 0.9783 - recall0.9: 0.6854 - accuracy: 0.9606 - auc: 0
.9727 - f1: 0.2624 - val_loss: 0.2832 - val_tp0.1: 1495971.0000 - val_fp0.1: 516145.0000 - val_tn0.1: 7731650.0000 - val_fn0.1: 86634.0000 - val_precision0.1: 0.7435 - val_recall0.1: 0.945
3 - val_tp0.3: 1379197.0000 - val_fp0.3: 185244.0000 - val_tn0.3: 8062551.0000 - val_fn0.3: 203408.0000 - val_precision0.3: 0.8816 - val_recall0.3: 0.8715 - val_tp0.5: 1284245.0000 - val_f
p0.5: 85898.0000 - val_tn0.5: 8161897.0000 - val_fn0.5: 298360.0000 - val_precision0.5: 0.9373 - val_recall0.5: 0.8115 - val_tp0.7: 1199963.0000 - val_fp0.7: 43094.0000 - val_tn0.7: 820470
1.0000 - val_fn0.7: 382642.0000 - val_precision0.7: 0.9653 - val_recall0.7: 0.7582 - val_tp0.9: 968381.0000 - val_fp0.9: 8568.0000 - val_tn0.9: 8239227.0000 - val_fn0.9: 614224.0000 - val_
precision0.9: 0.9912 - val_recall0.9: 0.6119 - val_accuracy: 0.9609 - val_auc: 0.9676 - val_f1: 0.2773
Epoch 12/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2531 - tp0.1: 5703005.0000 - fp0.1: 2614169.0000 - tn0.1: 30770672.0000 - fn0.1: 233771.0000 - precision0.1: 0.6857 - rec
all0.1: 0.9606 - tp0.3: 5460500.0000 - fp0.3: 1362191.0000 - tn0.3: 32022638.0000 - fn0.3: 476276.0000 - precision0.3: 0.8003 - recall0.3: 0.9198 - tp0.5: 5066536.0000 - fp0.5: 594270.0000
 - tn0.5: 32790576.0000 - fn0.5: 870240.0000 - precision0.5: 0.8950 - recall0.5: 0.8534 - tp0.7: 4782731.0000 - fp0.7: 324884.0000 - tn0.7: 33059918.0000 - fn0.7: 1154045.0000 - precision0
.7: 0.9364 - recall0.7: 0.8056 - tp0.9: 4134257.0000 - fp0.9: 91592.0000 - tn0.9: 33293228.0000 - fn0.9: 1802519.0000 - precision0.9: 0.9783 - recall0.9: 0.6964 - accuracy: 0.9628 - auc: 0
.9742 - f1: 0.2624 - val_loss: 0.2880 - val_tp0.1: 1541566.0000 - val_fp0.1: 930091.0000 - val_tn0.1: 7317704.0000 - val_fn0.1: 41039.0000 - val_precision0.1: 0.6237 - val_recall0.1: 0.974
1 - val_tp0.3: 1476199.0000 - val_fp0.3: 465005.0000 - val_tn0.3: 7782790.0000 - val_fn0.3: 106406.0000 - val_precision0.3: 0.7605 - val_recall0.3: 0.9328 - val_tp0.5: 1399471.0000 - val_f
p0.5: 255679.0000 - val_tn0.5: 7992116.0000 - val_fn0.5: 183134.0000 - val_precision0.5: 0.8455 - val_recall0.5: 0.8843 - val_tp0.7: 1343272.0000 - val_fp0.7: 156828.0000 - val_tn0.7: 8090
967.0000 - val_fn0.7: 239333.0000 - val_precision0.7: 0.8955 - val_recall0.7: 0.8488 - val_tp0.9: 1199567.0000 - val_fp0.9: 59158.0000 - val_tn0.9: 8188637.0000 - val_fn0.9: 383038.0000 -
val_precision0.9: 0.9530 - val_recall0.9: 0.7580 - val_accuracy: 0.9554 - val_auc: 0.9778 - val_f1: 0.2773
Epoch 13/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2610 - tp0.1: 5692281.0000 - fp0.1: 2715173.0000 - tn0.1: 30669648.0000 - fn0.1: 244495.0000 - precision0.1: 0.6771 - rec
all0.1: 0.9588 - tp0.3: 5444152.0000 - fp0.3: 1405545.0000 - tn0.3: 31979284.0000 - fn0.3: 492624.0000 - precision0.3: 0.7948 - recall0.3: 0.9170 - tp0.5: 5067232.0000 - fp0.5: 645126.0000
 - tn0.5: 32739708.0000 - fn0.5: 869544.0000 - precision0.5: 0.8871 - recall0.5: 0.8535 - tp0.7: 4741815.0000 - fp0.7: 326101.0000 - tn0.7: 33058726.0000 - fn0.7: 1194961.0000 - precision0
.7: 0.9357 - recall0.7: 0.7987 - tp0.9: 4095017.0000 - fp0.9: 90536.0000 - tn0.9: 33294300.0000 - fn0.9: 1841759.0000 - precision0.9: 0.9784 - recall0.9: 0.6898 - accuracy: 0.9615 - auc: 0
.9733 - f1: 0.2624 - val_loss: 0.3264 - val_tp0.1: 1552497.0000 - val_fp0.1: 1332139.0000 - val_tn0.1: 6915656.0000 - val_fn0.1: 30108.0000 - val_precision0.1: 0.5382 - val_recall0.1: 0.98
10 - val_tp0.3: 1491410.0000 - val_fp0.3: 604265.0000 - val_tn0.3: 7643530.0000 - val_fn0.3: 91195.0000 - val_precision0.3: 0.7117 - val_recall0.3: 0.9424 - val_tp0.5: 1423905.0000 - val_f
p0.5: 336821.0000 - val_tn0.5: 7910974.0000 - val_fn0.5: 158700.0000 - val_precision0.5: 0.8087 - val_recall0.5: 0.8997 - val_tp0.7: 1364436.0000 - val_fp0.7: 196850.0000 - val_tn0.7: 8050
945.0000 - val_fn0.7: 218169.0000 - val_precision0.7: 0.8739 - val_recall0.7: 0.8621 - val_tp0.9: 1214453.0000 - val_fp0.9: 61872.0000 - val_tn0.9: 8185923.0000 - val_fn0.9: 368152.0000 -
val_precision0.9: 0.9515 - val_recall0.9: 0.7674 - val_accuracy: 0.9496 - val_auc: 0.9794 - val_f1: 0.2773
Epoch 14/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2416 - tp0.1: 5720003.0000 - fp0.1: 2566509.0000 - tn0.1: 30818316.0000 - fn0.1: 216773.0000 - precision0.1: 0.6903 - rec
all0.1: 0.9635 - tp0.3: 5480611.0000 - fp0.3: 1304768.0000 - tn0.3: 32080068.0000 - fn0.3: 456165.0000 - precision0.3: 0.8077 - recall0.3: 0.9232 - tp0.5: 5185408.0000 - fp0.5: 690719.0000
 - tn0.5: 32694086.0000 - fn0.5: 751368.0000 - precision0.5: 0.8825 - recall0.5: 0.8734 - tp0.7: 4822441.0000 - fp0.7: 319155.0000 - tn0.7: 33065634.0000 - fn0.7: 1114335.0000 - precision0
.7: 0.9379 - recall0.7: 0.8123 - tp0.9: 4172445.0000 - fp0.9: 86620.0000 - tn0.9: 33298202.0000 - fn0.9: 1764331.0000 - precision0.9: 0.9797 - recall0.9: 0.7028 - accuracy: 0.9633 - auc: 0
.9762 - f1: 0.2624 - val_loss: 0.3018 - val_tp0.1: 1466734.0000 - val_fp0.1: 405231.0000 - val_tn0.1: 7842564.0000 - val_fn0.1: 115871.0000 - val_precision0.1: 0.7835 - val_recall0.1: 0.92
68 - val_tp0.3: 1381330.0000 - val_fp0.3: 201920.0000 - val_tn0.3: 8045875.0000 - val_fn0.3: 201275.0000 - val_precision0.3: 0.8725 - val_recall0.3: 0.8728 - val_tp0.5: 1281231.0000 - val_
fp0.5: 78305.0000 - val_tn0.5: 8169490.0000 - val_fn0.5: 301374.0000 - val_precision0.5: 0.9424 - val_recall0.5: 0.8096 - val_tp0.7: 1204999.0000 - val_fp0.7: 38474.0000 - val_tn0.7: 82093
21.0000 - val_fn0.7: 377606.0000 - val_precision0.7: 0.9691 - val_recall0.7: 0.7614 - val_tp0.9: 1069398.0000 - val_fp0.9: 12845.0000 - val_tn0.9: 8234950.0000 - val_fn0.9: 513207.0000 - v
al_precision0.9: 0.9881 - val_recall0.9: 0.6757 - val_accuracy: 0.9614 - val_auc: 0.9591 - val_f1: 0.2773
Epoch 15/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2295 - tp0.1: 5722432.0000 - fp0.1: 2370696.0000 - tn0.1: 31014116.0000 - fn0.1: 214344.0000 - precision0.1: 0.7071 - rec
all0.1: 0.9639 - tp0.3: 5503430.0000 - fp0.3: 1205741.0000 - tn0.3: 32179086.0000 - fn0.3: 433346.0000 - precision0.3: 0.8203 - recall0.3: 0.9270 - tp0.5: 5226789.0000 - fp0.5: 622292.0000
 - tn0.5: 32762524.0000 - fn0.5: 709987.0000 - precision0.5: 0.8936 - recall0.5: 0.8804 - tp0.7: 4882199.0000 - fp0.7: 299113.0000 - tn0.7: 33085700.0000 - fn0.7: 1054577.0000 - precision0
.7: 0.9423 - recall0.7: 0.8224 - tp0.9: 4261326.0000 - fp0.9: 78340.0000 - tn0.9: 33306484.0000 - fn0.9: 1675450.0000 - precision0.9: 0.9819 - recall0.9: 0.7178 - accuracy: 0.9661 - auc: 0
.9771 - f1: 0.2624 - val_loss: 0.2662 - val_tp0.1: 1543285.0000 - val_fp0.1: 895817.0000 - val_tn0.1: 7351978.0000 - val_fn0.1: 39320.0000 - val_precision0.1: 0.6327 - val_recall0.1: 0.975
2 - val_tp0.3: 1493466.0000 - val_fp0.3: 427226.0000 - val_tn0.3: 7820569.0000 - val_fn0.3: 89139.0000 - val_precision0.3: 0.7776 - val_recall0.3: 0.9437 - val_tp0.5: 1429946.0000 - val_fp
0.5: 249890.0000 - val_tn0.5: 7997905.0000 - val_fn0.5: 152659.0000 - val_precision0.5: 0.8512 - val_recall0.5: 0.9035 - val_tp0.7: 1335056.0000 - val_fp0.7: 132081.0000 - val_tn0.7: 81157
14.0000 - val_fn0.7: 247549.0000 - val_precision0.7: 0.9100 - val_recall0.7: 0.8436 - val_tp0.9: 1178764.0000 - val_fp0.9: 50269.0000 - val_tn0.9: 8197526.0000 - val_fn0.9: 403841.0000 - v
al_precision0.9: 0.9591 - val_recall0.9: 0.7448 - val_accuracy: 0.9591 - val_auc: 0.9796 - val_f1: 0.2773
Epoch 16/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2200 - tp0.1: 5727744.0000 - fp0.1: 2233811.0000 - tn0.1: 31151016.0000 - fn0.1: 209032.0000 - precision0.1: 0.7194 - rec
all0.1: 0.9648 - tp0.3: 5535079.0000 - fp0.3: 1172465.0000 - tn0.3: 32212368.0000 - fn0.3: 401697.0000 - precision0.3: 0.8252 - recall0.3: 0.9323 - tp0.5: 5279991.0000 - fp0.5: 630115.0000
 - tn0.5: 32754706.0000 - fn0.5: 656785.0000 - precision0.5: 0.8934 - recall0.5: 0.8894 - tp0.7: 4922795.0000 - fp0.7: 289713.0000 - tn0.7: 33095104.0000 - fn0.7: 1013981.0000 - precision0
.7: 0.9444 - recall0.7: 0.8292 - tp0.9: 4339994.0000 - fp0.9: 81950.0000 - tn0.9: 33302856.0000 - fn0.9: 1596782.0000 - precision0.9: 0.9815 - recall0.9: 0.7310 - accuracy: 0.9673 - auc: 0
.9780 - f1: 0.2624 - val_loss: 0.3052 - val_tp0.1: 1502027.0000 - val_fp0.1: 713668.0000 - val_tn0.1: 7534127.0000 - val_fn0.1: 80578.0000 - val_precision0.1: 0.6779 - val_recall0.1: 0.949
1 - val_tp0.3: 1373186.0000 - val_fp0.3: 249430.0000 - val_tn0.3: 7998365.0000 - val_fn0.3: 209419.0000 - val_precision0.3: 0.8463 - val_recall0.3: 0.8677 - val_tp0.5: 1287724.0000 - val_f
p0.5: 120697.0000 - val_tn0.5: 8127098.0000 - val_fn0.5: 294881.0000 - val_precision0.5: 0.9143 - val_recall0.5: 0.8137 - val_tp0.7: 1203991.0000 - val_fp0.7: 63408.0000 - val_tn0.7: 81843
87.0000 - val_fn0.7: 378614.0000 - val_precision0.7: 0.9500 - val_recall0.7: 0.7608 - val_tp0.9: 994204.0000 - val_fp0.9: 14550.0000 - val_tn0.9: 8233245.0000 - val_fn0.9: 588401.0000 - va
l_precision0.9: 0.9856 - val_recall0.9: 0.6282 - val_accuracy: 0.9577 - val_auc: 0.9681 - val_f1: 0.2773
Epoch 17/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2132 - tp0.1: 5740894.0000 - fp0.1: 2232275.0000 - tn0.1: 31152552.0000 - fn0.1: 195882.0000 - precision0.1: 0.7200 - rec
all0.1: 0.9670 - tp0.3: 5532271.0000 - fp0.3: 1168226.0000 - tn0.3: 32216606.0000 - fn0.3: 404505.0000 - precision0.3: 0.8257 - recall0.3: 0.9319 - tp0.5: 5273881.0000 - fp0.5: 609565.0000
 - tn0.5: 32775244.0000 - fn0.5: 662895.0000 - precision0.5: 0.8964 - recall0.5: 0.8883 - tp0.7: 4954897.0000 - fp0.7: 286255.0000 - tn0.7: 33098576.0000 - fn0.7: 981879.0000 - precision0.
7: 0.9454 - recall0.7: 0.8346 - tp0.9: 4392174.0000 - fp0.9: 78566.0000 - tn0.9: 33306278.0000 - fn0.9: 1544602.0000 - precision0.9: 0.9824 - recall0.9: 0.7398 - accuracy: 0.9676 - auc: 0.
9792 - f1: 0.2624 - val_loss: 0.2674 - val_tp0.1: 1542015.0000 - val_fp0.1: 886833.0000 - val_tn0.1: 7360962.0000 - val_fn0.1: 40590.0000 - val_precision0.1: 0.6349 - val_recall0.1: 0.9744
 - val_tp0.3: 1513414.0000 - val_fp0.3: 533995.0000 - val_tn0.3: 7713800.0000 - val_fn0.3: 69191.0000 - val_precision0.3: 0.7392 - val_recall0.3: 0.9563 - val_tp0.5: 1469868.0000 - val_fp0
.5: 320357.0000 - val_tn0.5: 7927438.0000 - val_fn0.5: 112737.0000 - val_precision0.5: 0.8211 - val_recall0.5: 0.9288 - val_tp0.7: 1389256.0000 - val_fp0.7: 180638.0000 - val_tn0.7: 806715
7.0000 - val_fn0.7: 193349.0000 - val_precision0.7: 0.8849 - val_recall0.7: 0.8778 - val_tp0.9: 1265894.0000 - val_fp0.9: 72673.0000 - val_tn0.9: 8175122.0000 - val_fn0.9: 316711.0000 - va
l_precision0.9: 0.9457 - val_recall0.9: 0.7999 - val_accuracy: 0.9559 - val_auc: 0.9797 - val_f1: 0.2773
Epoch 18/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2127 - tp0.1: 5744925.0000 - fp0.1: 2262684.0000 - tn0.1: 31122132.0000 - fn0.1: 191851.0000 - precision0.1: 0.7174 - rec
all0.1: 0.9677 - tp0.3: 5540864.0000 - fp0.3: 1151165.0000 - tn0.3: 32233664.0000 - fn0.3: 395912.0000 - precision0.3: 0.8280 - recall0.3: 0.9333 - tp0.5: 5275650.0000 - fp0.5: 595986.0000
 - tn0.5: 32788824.0000 - fn0.5: 661126.0000 - precision0.5: 0.8985 - recall0.5: 0.8886 - tp0.7: 4961721.0000 - fp0.7: 298376.0000 - tn0.7: 33086462.0000 - fn0.7: 975055.0000 - precision0.
7: 0.9433 - recall0.7: 0.8358 - tp0.9: 4369101.0000 - fp0.9: 83285.0000 - tn0.9: 33301528.0000 - fn0.9: 1567675.0000 - precision0.9: 0.9813 - recall0.9: 0.7359 - accuracy: 0.9680 - auc: 0.
9797 - f1: 0.2624 - val_loss: 0.2642 - val_tp0.1: 1498157.0000 - val_fp0.1: 419273.0000 - val_tn0.1: 7828522.0000 - val_fn0.1: 84448.0000 - val_precision0.1: 0.7813 - val_recall0.1: 0.9466
 - val_tp0.3: 1376765.0000 - val_fp0.3: 147925.0000 - val_tn0.3: 8099870.0000 - val_fn0.3: 205840.0000 - val_precision0.3: 0.9030 - val_recall0.3: 0.8699 - val_tp0.5: 1282934.0000 - val_fp
0.5: 68148.0000 - val_tn0.5: 8179647.0000 - val_fn0.5: 299671.0000 - val_precision0.5: 0.9496 - val_recall0.5: 0.8106 - val_tp0.7: 1192758.0000 - val_fp0.7: 34480.0000 - val_tn0.7: 8213315
.0000 - val_fn0.7: 389847.0000 - val_precision0.7: 0.9719 - val_recall0.7: 0.7537 - val_tp0.9: 940508.0000 - val_fp0.9: 5969.0000 - val_tn0.9: 8241826.0000 - val_fn0.9: 642097.0000 - val_p
recision0.9: 0.9937 - val_recall0.9: 0.5943 - val_accuracy: 0.9626 - val_auc: 0.9698 - val_f1: 0.2773
Epoch 19/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2024 - tp0.1: 5743078.0000 - fp0.1: 2054017.0000 - tn0.1: 31330796.0000 - fn0.1: 193698.0000 - precision0.1: 0.7366 - rec
all0.1: 0.9674 - tp0.3: 5562740.0000 - fp0.3: 1087818.0000 - tn0.3: 32297020.0000 - fn0.3: 374036.0000 - precision0.3: 0.8364 - recall0.3: 0.9370 - tp0.5: 5320915.0000 - fp0.5: 581625.0000
 - tn0.5: 32803212.0000 - fn0.5: 615861.0000 - precision0.5: 0.9015 - recall0.5: 0.8963 - tp0.7: 5014211.0000 - fp0.7: 278329.0000 - tn0.7: 33106496.0000 - fn0.7: 922565.0000 - precision0.
7: 0.9474 - recall0.7: 0.8446 - tp0.9: 4439455.0000 - fp0.9: 69924.0000 - tn0.9: 33314888.0000 - fn0.9: 1497321.0000 - precision0.9: 0.9845 - recall0.9: 0.7478 - accuracy: 0.9695 - auc: 0.
9800 - f1: 0.2624 - val_loss: 0.2474 - val_tp0.1: 1500397.0000 - val_fp0.1: 388201.0000 - val_tn0.1: 7859594.0000 - val_fn0.1: 82208.0000 - val_precision0.1: 0.7945 - val_recall0.1: 0.9481
 - val_tp0.3: 1417832.0000 - val_fp0.3: 177501.0000 - val_tn0.3: 8070294.0000 - val_fn0.3: 164773.0000 - val_precision0.3: 0.8887 - val_recall0.3: 0.8959 - val_tp0.5: 1324841.0000 - val_fp
0.5: 83172.0000 - val_tn0.5: 8164623.0000 - val_fn0.5: 257764.0000 - val_precision0.5: 0.9409 - val_recall0.5: 0.8371 - val_tp0.7: 1241641.0000 - val_fp0.7: 44194.0000 - val_tn0.7: 8203601
.0000 - val_fn0.7: 340964.0000 - val_precision0.7: 0.9656 - val_recall0.7: 0.7846 - val_tp0.9: 1008204.0000 - val_fp0.9: 8267.0000 - val_tn0.9: 8239528.0000 - val_fn0.9: 574401.0000 - val_
precision0.9: 0.9919 - val_recall0.9: 0.6371 - val_accuracy: 0.9653 - val_auc: 0.9703 - val_f1: 0.2773
Epoch 20/20
480/480 [==============================] - 21s 44ms/step - loss: 0.1967 - tp0.1: 5751226.0000 - fp0.1: 2003963.0000 - tn0.1: 31380856.0000 - fn0.1: 185550.0000 - precision0.1: 0.7416 - rec
all0.1: 0.9687 - tp0.3: 5579827.0000 - fp0.3: 1068977.0000 - tn0.3: 32315852.0000 - fn0.3: 356949.0000 - precision0.3: 0.8392 - recall0.3: 0.9399 - tp0.5: 5349645.0000 - fp0.5: 581039.0000
 - tn0.5: 32803788.0000 - fn0.5: 587131.0000 - precision0.5: 0.9020 - recall0.5: 0.9011 - tp0.7: 5051728.0000 - fp0.7: 282220.0000 - tn0.7: 33102590.0000 - fn0.7: 885048.0000 - precision0.
7: 0.9471 - recall0.7: 0.8509 - tp0.9: 4482265.0000 - fp0.9: 74224.0000 - tn0.9: 33310598.0000 - fn0.9: 1454511.0000 - precision0.9: 0.9837 - recall0.9: 0.7550 - accuracy: 0.9703 - auc: 0.
9808 - f1: 0.2624 - val_loss: 0.2503 - val_tp0.1: 1534044.0000 - val_fp0.1: 725110.0000 - val_tn0.1: 7522685.0000 - val_fn0.1: 48561.0000 - val_precision0.1: 0.6790 - val_recall0.1: 0.9693
 - val_tp0.3: 1480024.0000 - val_fp0.3: 353833.0000 - val_tn0.3: 7893962.0000 - val_fn0.3: 102581.0000 - val_precision0.3: 0.8071 - val_recall0.3: 0.9352 - val_tp0.5: 1407828.0000 - val_fp
0.5: 193567.0000 - val_tn0.5: 8054228.0000 - val_fn0.5: 174777.0000 - val_precision0.5: 0.8791 - val_recall0.5: 0.8896 - val_tp0.7: 1340054.0000 - val_fp0.7: 119720.0000 - val_tn0.7: 81280
75.0000 - val_fn0.7: 242551.0000 - val_precision0.7: 0.9180 - val_recall0.7: 0.8467 - val_tp0.9: 1179562.0000 - val_fp0.9: 41420.0000 - val_tn0.9: 8206375.0000 - val_fn0.9: 403043.0000 - v
al_precision0.9: 0.9661 - val_recall0.9: 0.7453 - val_accuracy: 0.9625 - val_auc: 0.9780 - val_f1: 0.2773
--- Running training session 56/140
{'hp_epochs': 20, 'hp_batch_size': 10, 'hp_scaler': 'robust', 'hp_n_levels': 5, 'hp_first_filters': 16, 'hp_pool_size': 4, 'hp_input_size': 8192, 'hp_lr_start': 0.06276763366515732, 'hp_lr
_power': 1.0}
--- repeat #: 2
input - shape:   (None, 8192, 1)
output - shape:  (None, 8192, 1)
Epoch 1/20
480/480 [==============================] - 44s 52ms/step - loss: 0.6448 - tp0.1: 5265662.0000 - fp0.1: 6246252.0000 - tn0.1: 27138564.0000 - fn0.1: 671114.0000 - precision0.1: 0.4574 - rec
all0.1: 0.8870 - tp0.3: 4710949.0000 - fp0.3: 3395106.0000 - tn0.3: 29989722.0000 - fn0.3: 1225827.0000 - precision0.3: 0.5812 - recall0.3: 0.7935 - tp0.5: 3693283.0000 - fp0.5: 1049113.00
00 - tn0.5: 32335700.0000 - fn0.5: 2243493.0000 - precision0.5: 0.7788 - recall0.5: 0.6221 - tp0.7: 3063860.0000 - fp0.7: 405953.0000 - tn0.7: 32978848.0000 - fn0.7: 2872916.0000 - precisi
on0.7: 0.8830 - recall0.7: 0.5161 - tp0.9: 2353054.0000 - fp0.9: 100482.0000 - tn0.9: 33284332.0000 - fn0.9: 3583722.0000 - precision0.9: 0.9590 - recall0.9: 0.3964 - accuracy: 0.9163 - au
c: 0.9133 - f1: 0.2624 - val_loss: 1.3199 - val_tp0.1: 1131372.0000 - val_fp0.1: 737035.0000 - val_tn0.1: 7510760.0000 - val_fn0.1: 451233.0000 - val_precision0.1: 0.6055 - val_recall0.1:
0.7149 - val_tp0.3: 1034683.0000 - val_fp0.3: 330216.0000 - val_tn0.3: 7917579.0000 - val_fn0.3: 547922.0000 - val_precision0.3: 0.7581 - val_recall0.3: 0.6538 - val_tp0.5: 928465.0000 - v
al_fp0.5: 121376.0000 - val_tn0.5: 8126419.0000 - val_fn0.5: 654140.0000 - val_precision0.5: 0.8844 - val_recall0.5: 0.5867 - val_tp0.7: 829955.0000 - val_fp0.7: 60010.0000 - val_tn0.7: 81
87785.0000 - val_fn0.7: 752650.0000 - val_precision0.7: 0.9326 - val_recall0.7: 0.5244 - val_tp0.9: 673776.0000 - val_fp0.9: 11090.0000 - val_tn0.9: 8236705.0000 - val_fn0.9: 908829.0000 -
 val_precision0.9: 0.9838 - val_recall0.9: 0.4257 - val_accuracy: 0.9211 - val_auc: 0.8400 - val_f1: 0.2773
Epoch 2/20
480/480 [==============================] - 21s 44ms/step - loss: 0.5271 - tp0.1: 5418063.0000 - fp0.1: 5526907.0000 - tn0.1: 27857926.0000 - fn0.1: 518713.0000 - precision0.1: 0.4950 - rec
all0.1: 0.9126 - tp0.3: 4862070.0000 - fp0.3: 2474103.0000 - tn0.3: 30910734.0000 - fn0.3: 1074706.0000 - precision0.3: 0.6628 - recall0.3: 0.8190 - tp0.5: 4124120.0000 - fp0.5: 1036855.00
00 - tn0.5: 32347980.0000 - fn0.5: 1812656.0000 - precision0.5: 0.7991 - recall0.5: 0.6947 - tp0.7: 3446425.0000 - fp0.7: 437082.0000 - tn0.7: 32947768.0000 - fn0.7: 2490351.0000 - precisi
on0.7: 0.8875 - recall0.7: 0.5805 - tp0.9: 2633706.0000 - fp0.9: 118948.0000 - tn0.9: 33265864.0000 - fn0.9: 3303070.0000 - precision0.9: 0.9568 - recall0.9: 0.4436 - accuracy: 0.9275 - au
c: 0.9331 - f1: 0.2624 - val_loss: 1.1481 - val_tp0.1: 991975.0000 - val_fp0.1: 327918.0000 - val_tn0.1: 7919877.0000 - val_fn0.1: 590630.0000 - val_precision0.1: 0.7516 - val_recall0.1: 0
.6268 - val_tp0.3: 921560.0000 - val_fp0.3: 203493.0000 - val_tn0.3: 8044302.0000 - val_fn0.3: 661045.0000 - val_precision0.3: 0.8191 - val_recall0.3: 0.5823 - val_tp0.5: 816746.0000 - val
_fp0.5: 96370.0000 - val_tn0.5: 8151425.0000 - val_fn0.5: 765859.0000 - val_precision0.5: 0.8945 - val_recall0.5: 0.5161 - val_tp0.7: 638180.0000 - val_fp0.7: 16690.0000 - val_tn0.7: 82311
05.0000 - val_fn0.7: 944425.0000 - val_precision0.7: 0.9745 - val_recall0.7: 0.4032 - val_tp0.9: 454222.0000 - val_fp0.9: 1300.0000 - val_tn0.9: 8246495.0000 - val_fn0.9: 1128383.0000 - va
l_precision0.9: 0.9971 - val_recall0.9: 0.2870 - val_accuracy: 0.9123 - val_auc: 0.8046 - val_f1: 0.2773
Epoch 3/20
480/480 [==============================] - 21s 44ms/step - loss: 0.4694 - tp0.1: 5506389.0000 - fp0.1: 5279352.0000 - tn0.1: 28105468.0000 - fn0.1: 430387.0000 - precision0.1: 0.5105 - rec
all0.1: 0.9275 - tp0.3: 4935930.0000 - fp0.3: 2185008.0000 - tn0.3: 31199836.0000 - fn0.3: 1000846.0000 - precision0.3: 0.6932 - recall0.3: 0.8314 - tp0.5: 4402685.0000 - fp0.5: 1013988.00
00 - tn0.5: 32370838.0000 - fn0.5: 1534091.0000 - precision0.5: 0.8128 - recall0.5: 0.7416 - tp0.7: 3775055.0000 - fp0.7: 446059.0000 - tn0.7: 32938768.0000 - fn0.7: 2161721.0000 - precisi
on0.7: 0.8943 - recall0.7: 0.6359 - tp0.9: 2837543.0000 - fp0.9: 112976.0000 - tn0.9: 33271836.0000 - fn0.9: 3099233.0000 - precision0.9: 0.9617 - recall0.9: 0.4780 - accuracy: 0.9352 - au
c: 0.9432 - f1: 0.2624 - val_loss: 1.1726 - val_tp0.1: 958675.0000 - val_fp0.1: 396142.0000 - val_tn0.1: 7851653.0000 - val_fn0.1: 623930.0000 - val_precision0.1: 0.7076 - val_recall0.1: 0
.6058 - val_tp0.3: 888712.0000 - val_fp0.3: 224696.0000 - val_tn0.3: 8023099.0000 - val_fn0.3: 693893.0000 - val_precision0.3: 0.7982 - val_recall0.3: 0.5616 - val_tp0.5: 821633.0000 - val
_fp0.5: 102326.0000 - val_tn0.5: 8145469.0000 - val_fn0.5: 760972.0000 - val_precision0.5: 0.8893 - val_recall0.5: 0.5192 - val_tp0.7: 735292.0000 - val_fp0.7: 39372.0000 - val_tn0.7: 8208
423.0000 - val_fn0.7: 847313.0000 - val_precision0.7: 0.9492 - val_recall0.7: 0.4646 - val_tp0.9: 572482.0000 - val_fp0.9: 1157.0000 - val_tn0.9: 8246638.0000 - val_fn0.9: 1010123.0000 - v
al_precision0.9: 0.9980 - val_recall0.9: 0.3617 - val_accuracy: 0.9122 - val_auc: 0.7943 - val_f1: 0.2773
Epoch 4/20
480/480 [==============================] - 21s 43ms/step - loss: 0.4326 - tp0.1: 5560029.0000 - fp0.1: 4926737.0000 - tn0.1: 28458088.0000 - fn0.1: 376747.0000 - precision0.1: 0.5302 - rec
all0.1: 0.9365 - tp0.3: 5043034.0000 - fp0.3: 2165200.0000 - tn0.3: 31219610.0000 - fn0.3: 893742.0000 - precision0.3: 0.6996 - recall0.3: 0.8495 - tp0.5: 4510820.0000 - fp0.5: 990813.0000
 - tn0.5: 32394014.0000 - fn0.5: 1425956.0000 - precision0.5: 0.8199 - recall0.5: 0.7598 - tp0.7: 3929915.0000 - fp0.7: 460992.0000 - tn0.7: 32923836.0000 - fn0.7: 2006861.0000 - precision
0.7: 0.8950 - recall0.7: 0.6620 - tp0.9: 2983151.0000 - fp0.9: 124501.0000 - tn0.9: 33260320.0000 - fn0.9: 2953625.0000 - precision0.9: 0.9599 - recall0.9: 0.5025 - accuracy: 0.9385 - auc:
 0.9491 - f1: 0.2624 - val_loss: 0.4556 - val_tp0.1: 1447920.0000 - val_fp0.1: 910577.0000 - val_tn0.1: 7337218.0000 - val_fn0.1: 134685.0000 - val_precision0.1: 0.6139 - val_recall0.1: 0.
9149 - val_tp0.3: 1348323.0000 - val_fp0.3: 526913.0000 - val_tn0.3: 7720882.0000 - val_fn0.3: 234282.0000 - val_precision0.3: 0.7190 - val_recall0.3: 0.8520 - val_tp0.5: 1171829.0000 - va
l_fp0.5: 254087.0000 - val_tn0.5: 7993708.0000 - val_fn0.5: 410776.0000 - val_precision0.5: 0.8218 - val_recall0.5: 0.7404 - val_tp0.7: 1027826.0000 - val_fp0.7: 124376.0000 - val_tn0.7: 8
123419.0000 - val_fn0.7: 554779.0000 - val_precision0.7: 0.8921 - val_recall0.7: 0.6495 - val_tp0.9: 747344.0000 - val_fp0.9: 3170.0000 - val_tn0.9: 8244625.0000 - val_fn0.9: 835261.0000 -
 val_precision0.9: 0.9958 - val_recall0.9: 0.4722 - val_accuracy: 0.9324 - val_auc: 0.9418 - val_f1: 0.2773
Epoch 5/20
480/480 [==============================] - 21s 44ms/step - loss: 0.4141 - tp0.1: 5574699.0000 - fp0.1: 4604743.0000 - tn0.1: 28780080.0000 - fn0.1: 362077.0000 - precision0.1: 0.5476 - rec
all0.1: 0.9390 - tp0.3: 5097801.0000 - fp0.3: 2138526.0000 - tn0.3: 31246316.0000 - fn0.3: 838975.0000 - precision0.3: 0.7045 - recall0.3: 0.8587 - tp0.5: 4556087.0000 - fp0.5: 942923.0000
 - tn0.5: 32441904.0000 - fn0.5: 1380689.0000 - precision0.5: 0.8285 - recall0.5: 0.7674 - tp0.7: 3978370.0000 - fp0.7: 450803.0000 - tn0.7: 32934030.0000 - fn0.7: 1958406.0000 - precision
0.7: 0.8982 - recall0.7: 0.6701 - tp0.9: 3000212.0000 - fp0.9: 125980.0000 - tn0.9: 33258844.0000 - fn0.9: 2936564.0000 - precision0.9: 0.9597 - recall0.9: 0.5054 - accuracy: 0.9409 - auc:
 0.9524 - f1: 0.2624 - val_loss: 0.4822 - val_tp0.1: 1476924.0000 - val_fp0.1: 1123072.0000 - val_tn0.1: 7124723.0000 - val_fn0.1: 105681.0000 - val_precision0.1: 0.5680 - val_recall0.1: 0
.9332 - val_tp0.3: 1253583.0000 - val_fp0.3: 295322.0000 - val_tn0.3: 7952473.0000 - val_fn0.3: 329022.0000 - val_precision0.3: 0.8093 - val_recall0.3: 0.7921 - val_tp0.5: 1023581.0000 - v
al_fp0.5: 50787.0000 - val_tn0.5: 8197008.0000 - val_fn0.5: 559024.0000 - val_precision0.5: 0.9527 - val_recall0.5: 0.6468 - val_tp0.7: 804131.0000 - val_fp0.7: 19006.0000 - val_tn0.7: 822
8789.0000 - val_fn0.7: 778474.0000 - val_precision0.7: 0.9769 - val_recall0.7: 0.5081 - val_tp0.9: 623158.0000 - val_fp0.9: 3898.0000 - val_tn0.9: 8243897.0000 - val_fn0.9: 959447.0000 - v
al_precision0.9: 0.9938 - val_recall0.9: 0.3938 - val_accuracy: 0.9380 - val_auc: 0.9508 - val_f1: 0.2773
Epoch 6/20
480/480 [==============================] - 21s 44ms/step - loss: 0.3607 - tp0.1: 5623701.0000 - fp0.1: 3956775.0000 - tn0.1: 29428044.0000 - fn0.1: 313075.0000 - precision0.1: 0.5870 - rec
all0.1: 0.9473 - tp0.3: 5201616.0000 - fp0.3: 1881599.0000 - tn0.3: 31503236.0000 - fn0.3: 735160.0000 - precision0.3: 0.7344 - recall0.3: 0.8762 - tp0.5: 4749347.0000 - fp0.5: 857493.0000
 - tn0.5: 32527328.0000 - fn0.5: 1187429.0000 - precision0.5: 0.8471 - recall0.5: 0.8000 - tp0.7: 4289257.0000 - fp0.7: 431585.0000 - tn0.7: 32953252.0000 - fn0.7: 1647519.0000 - precision
0.7: 0.9086 - recall0.7: 0.7225 - tp0.9: 3310664.0000 - fp0.9: 120910.0000 - tn0.9: 33263930.0000 - fn0.9: 2626112.0000 - precision0.9: 0.9648 - recall0.9: 0.5577 - accuracy: 0.9480 - auc:
 0.9600 - f1: 0.2624 - val_loss: 0.3199 - val_tp0.1: 1487331.0000 - val_fp0.1: 694274.0000 - val_tn0.1: 7553521.0000 - val_fn0.1: 95274.0000 - val_precision0.1: 0.6818 - val_recall0.1: 0.9
398 - val_tp0.3: 1386296.0000 - val_fp0.3: 286146.0000 - val_tn0.3: 7961649.0000 - val_fn0.3: 196309.0000 - val_precision0.3: 0.8289 - val_recall0.3: 0.8760 - val_tp0.5: 1304018.0000 - val
_fp0.5: 139220.0000 - val_tn0.5: 8108575.0000 - val_fn0.5: 278587.0000 - val_precision0.5: 0.9035 - val_recall0.5: 0.8240 - val_tp0.7: 1235725.0000 - val_fp0.7: 76207.0000 - val_tn0.7: 817
1588.0000 - val_fn0.7: 346880.0000 - val_precision0.7: 0.9419 - val_recall0.7: 0.7808 - val_tp0.9: 1080668.0000 - val_fp0.9: 23195.0000 - val_tn0.9: 8224600.0000 - val_fn0.9: 501937.0000 -
 val_precision0.9: 0.9790 - val_recall0.9: 0.6828 - val_accuracy: 0.9575 - val_auc: 0.9627 - val_f1: 0.2773
Epoch 7/20
480/480 [==============================] - 21s 43ms/step - loss: 0.3314 - tp0.1: 5646234.0000 - fp0.1: 3554455.0000 - tn0.1: 29830372.0000 - fn0.1: 290542.0000 - precision0.1: 0.6137 - rec
all0.1: 0.9511 - tp0.3: 5227655.0000 - fp0.3: 1614171.0000 - tn0.3: 31770668.0000 - fn0.3: 709121.0000 - precision0.3: 0.7641 - recall0.3: 0.8806 - tp0.5: 4830553.0000 - fp0.5: 751013.0000
 - tn0.5: 32633820.0000 - fn0.5: 1106223.0000 - precision0.5: 0.8654 - recall0.5: 0.8137 - tp0.7: 4458898.0000 - fp0.7: 401272.0000 - tn0.7: 32983556.0000 - fn0.7: 1477878.0000 - precision
0.7: 0.9174 - recall0.7: 0.7511 - tp0.9: 3589678.0000 - fp0.9: 117919.0000 - tn0.9: 33266924.0000 - fn0.9: 2347098.0000 - precision0.9: 0.9682 - recall0.9: 0.6047 - accuracy: 0.9528 - auc:
 0.9641 - f1: 0.2624 - val_loss: 0.3516 - val_tp0.1: 1443246.0000 - val_fp0.1: 388491.0000 - val_tn0.1: 7859304.0000 - val_fn0.1: 139359.0000 - val_precision0.1: 0.7879 - val_recall0.1: 0.
9119 - val_tp0.3: 1363949.0000 - val_fp0.3: 220386.0000 - val_tn0.3: 8027409.0000 - val_fn0.3: 218656.0000 - val_precision0.3: 0.8609 - val_recall0.3: 0.8618 - val_tp0.5: 1277387.0000 - va
l_fp0.5: 127462.0000 - val_tn0.5: 8120333.0000 - val_fn0.5: 305218.0000 - val_precision0.5: 0.9093 - val_recall0.5: 0.8071 - val_tp0.7: 1201550.0000 - val_fp0.7: 76391.0000 - val_tn0.7: 81
71404.0000 - val_fn0.7: 381055.0000 - val_precision0.7: 0.9402 - val_recall0.7: 0.7592 - val_tp0.9: 1034837.0000 - val_fp0.9: 29131.0000 - val_tn0.9: 8218664.0000 - val_fn0.9: 547768.0000
- val_precision0.9: 0.9726 - val_recall0.9: 0.6539 - val_accuracy: 0.9560 - val_auc: 0.9517 - val_f1: 0.2773
Epoch 8/20
480/480 [==============================] - 21s 43ms/step - loss: 0.3124 - tp0.1: 5661924.0000 - fp0.1: 3284244.0000 - tn0.1: 30100560.0000 - fn0.1: 274852.0000 - precision0.1: 0.6329 - rec
all0.1: 0.9537 - tp0.3: 5341190.0000 - fp0.3: 1750123.0000 - tn0.3: 31634684.0000 - fn0.3: 595586.0000 - precision0.3: 0.7532 - recall0.3: 0.8997 - tp0.5: 4855283.0000 - fp0.5: 738314.0000
 - tn0.5: 32646522.0000 - fn0.5: 1081493.0000 - precision0.5: 0.8680 - recall0.5: 0.8178 - tp0.7: 4484249.0000 - fp0.7: 391982.0000 - tn0.7: 32992836.0000 - fn0.7: 1452527.0000 - precision
0.7: 0.9196 - recall0.7: 0.7553 - tp0.9: 3662123.0000 - fp0.9: 116143.0000 - tn0.9: 33268688.0000 - fn0.9: 2274653.0000 - precision0.9: 0.9693 - recall0.9: 0.6169 - accuracy: 0.9537 - auc:
 0.9668 - f1: 0.2624 - val_loss: 0.3804 - val_tp0.1: 1538470.0000 - val_fp0.1: 1344786.0000 - val_tn0.1: 6903009.0000 - val_fn0.1: 44135.0000 - val_precision0.1: 0.5336 - val_recall0.1: 0.
9721 - val_tp0.3: 1492276.0000 - val_fp0.3: 750571.0000 - val_tn0.3: 7497224.0000 - val_fn0.3: 90329.0000 - val_precision0.3: 0.6653 - val_recall0.3: 0.9429 - val_tp0.5: 1415057.0000 - val
_fp0.5: 403390.0000 - val_tn0.5: 7844405.0000 - val_fn0.5: 167548.0000 - val_precision0.5: 0.7782 - val_recall0.5: 0.8941 - val_tp0.7: 1355676.0000 - val_fp0.7: 251522.0000 - val_tn0.7: 79
96273.0000 - val_fn0.7: 226929.0000 - val_precision0.7: 0.8435 - val_recall0.7: 0.8566 - val_tp0.9: 1239605.0000 - val_fp0.9: 115484.0000 - val_tn0.9: 8132311.0000 - val_fn0.9: 343000.0000
 - val_precision0.9: 0.9148 - val_recall0.9: 0.7833 - val_accuracy: 0.9419 - val_auc: 0.9731 - val_f1: 0.2773
Epoch 9/20
480/480 [==============================] - 21s 43ms/step - loss: 0.3173 - tp0.1: 5659382.0000 - fp0.1: 3425500.0000 - tn0.1: 29959316.0000 - fn0.1: 277394.0000 - precision0.1: 0.6229 - rec
all0.1: 0.9533 - tp0.3: 5292646.0000 - fp0.3: 1671784.0000 - tn0.3: 31713034.0000 - fn0.3: 644130.0000 - precision0.3: 0.7600 - recall0.3: 0.8915 - tp0.5: 4877061.0000 - fp0.5: 747610.0000
 - tn0.5: 32637216.0000 - fn0.5: 1059715.0000 - precision0.5: 0.8671 - recall0.5: 0.8215 - tp0.7: 4496117.0000 - fp0.7: 402814.0000 - tn0.7: 32982008.0000 - fn0.7: 1440659.0000 - precision
0.7: 0.9178 - recall0.7: 0.7573 - tp0.9: 3666370.0000 - fp0.9: 116952.0000 - tn0.9: 33267872.0000 - fn0.9: 2270406.0000 - precision0.9: 0.9691 - recall0.9: 0.6176 - accuracy: 0.9540 - auc:
 0.9663 - f1: 0.2624 - val_loss: 0.4111 - val_tp0.1: 1461116.0000 - val_fp0.1: 755222.0000 - val_tn0.1: 7492573.0000 - val_fn0.1: 121489.0000 - val_precision0.1: 0.6592 - val_recall0.1: 0.
9232 - val_tp0.3: 1223139.0000 - val_fp0.3: 230361.0000 - val_tn0.3: 8017434.0000 - val_fn0.3: 359466.0000 - val_precision0.3: 0.8415 - val_recall0.3: 0.7729 - val_tp0.5: 1066667.0000 - va
l_fp0.5: 102255.0000 - val_tn0.5: 8145540.0000 - val_fn0.5: 515938.0000 - val_precision0.5: 0.9125 - val_recall0.5: 0.6740 - val_tp0.7: 886969.0000 - val_fp0.7: 41575.0000 - val_tn0.7: 820
6220.0000 - val_fn0.7: 695636.0000 - val_precision0.7: 0.9552 - val_recall0.7: 0.5604 - val_tp0.9: 665393.0000 - val_fp0.9: 8132.0000 - val_tn0.9: 8239663.0000 - val_fn0.9: 917212.0000 - v
al_precision0.9: 0.9879 - val_recall0.9: 0.4204 - val_accuracy: 0.9371 - val_auc: 0.9500 - val_f1: 0.2773
Epoch 10/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2749 - tp0.1: 5674290.0000 - fp0.1: 2738682.0000 - tn0.1: 30646168.0000 - fn0.1: 262486.0000 - precision0.1: 0.6745 - rec
all0.1: 0.9558 - tp0.3: 5383213.0000 - fp0.3: 1364871.0000 - tn0.3: 32019970.0000 - fn0.3: 553563.0000 - precision0.3: 0.7977 - recall0.3: 0.9068 - tp0.5: 4998441.0000 - fp0.5: 611762.0000
 - tn0.5: 32773056.0000 - fn0.5: 938335.0000 - precision0.5: 0.8910 - recall0.5: 0.8419 - tp0.7: 4697294.0000 - fp0.7: 334198.0000 - tn0.7: 33050624.0000 - fn0.7: 1239482.0000 - precision0
.7: 0.9336 - recall0.7: 0.7912 - tp0.9: 4025561.0000 - fp0.9: 94510.0000 - tn0.9: 33290300.0000 - fn0.9: 1911215.0000 - precision0.9: 0.9771 - recall0.9: 0.6781 - accuracy: 0.9606 - auc: 0
.9710 - f1: 0.2624 - val_loss: 0.3147 - val_tp0.1: 1502641.0000 - val_fp0.1: 644566.0000 - val_tn0.1: 7603229.0000 - val_fn0.1: 79964.0000 - val_precision0.1: 0.6998 - val_recall0.1: 0.949
5 - val_tp0.3: 1470125.0000 - val_fp0.3: 446059.0000 - val_tn0.3: 7801736.0000 - val_fn0.3: 112480.0000 - val_precision0.3: 0.7672 - val_recall0.3: 0.9289 - val_tp0.5: 1426341.0000 - val_f
p0.5: 321683.0000 - val_tn0.5: 7926112.0000 - val_fn0.5: 156264.0000 - val_precision0.5: 0.8160 - val_recall0.5: 0.9013 - val_tp0.7: 1390033.0000 - val_fp0.7: 241453.0000 - val_tn0.7: 8006
342.0000 - val_fn0.7: 192572.0000 - val_precision0.7: 0.8520 - val_recall0.7: 0.8783 - val_tp0.9: 1306070.0000 - val_fp0.9: 125298.0000 - val_tn0.9: 8122497.0000 - val_fn0.9: 276535.0000 -
 val_precision0.9: 0.9125 - val_recall0.9: 0.8253 - val_accuracy: 0.9514 - val_auc: 0.9668 - val_f1: 0.2773
Epoch 11/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2571 - tp0.1: 5701636.0000 - fp0.1: 2654470.0000 - tn0.1: 30730342.0000 - fn0.1: 235140.0000 - precision0.1: 0.6823 - rec
all0.1: 0.9604 - tp0.3: 5431401.0000 - fp0.3: 1345161.0000 - tn0.3: 32039650.0000 - fn0.3: 505375.0000 - precision0.3: 0.8015 - recall0.3: 0.9149 - tp0.5: 5052982.0000 - fp0.5: 600086.0000
 - tn0.5: 32784728.0000 - fn0.5: 883794.0000 - precision0.5: 0.8938 - recall0.5: 0.8511 - tp0.7: 4767820.0000 - fp0.7: 330196.0000 - tn0.7: 33054604.0000 - fn0.7: 1168956.0000 - precision0
.7: 0.9352 - recall0.7: 0.8031 - tp0.9: 4129243.0000 - fp0.9: 91729.0000 - tn0.9: 33293112.0000 - fn0.9: 1807533.0000 - precision0.9: 0.9783 - recall0.9: 0.6955 - accuracy: 0.9623 - auc: 0
.9739 - f1: 0.2624 - val_loss: 0.2934 - val_tp0.1: 1497012.0000 - val_fp0.1: 580336.0000 - val_tn0.1: 7667459.0000 - val_fn0.1: 85593.0000 - val_precision0.1: 0.7206 - val_recall0.1: 0.945
9 - val_tp0.3: 1418177.0000 - val_fp0.3: 304844.0000 - val_tn0.3: 7942951.0000 - val_fn0.3: 164428.0000 - val_precision0.3: 0.8231 - val_recall0.3: 0.8961 - val_tp0.5: 1344607.0000 - val_f
p0.5: 177499.0000 - val_tn0.5: 8070296.0000 - val_fn0.5: 237998.0000 - val_precision0.5: 0.8834 - val_recall0.5: 0.8496 - val_tp0.7: 1277773.0000 - val_fp0.7: 107504.0000 - val_tn0.7: 8140
291.0000 - val_fn0.7: 304832.0000 - val_precision0.7: 0.9224 - val_recall0.7: 0.8074 - val_tp0.9: 1107705.0000 - val_fp0.9: 32564.0000 - val_tn0.9: 8215231.0000 - val_fn0.9: 474900.0000 -
val_precision0.9: 0.9714 - val_recall0.9: 0.6999 - val_accuracy: 0.9577 - val_auc: 0.9663 - val_f1: 0.2773
Epoch 12/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2581 - tp0.1: 5693123.0000 - fp0.1: 2595417.0000 - tn0.1: 30789420.0000 - fn0.1: 243653.0000 - precision0.1: 0.6869 - rec
all0.1: 0.9590 - tp0.3: 5446545.0000 - fp0.3: 1373916.0000 - tn0.3: 32010920.0000 - fn0.3: 490231.0000 - precision0.3: 0.7986 - recall0.3: 0.9174 - tp0.5: 5065870.0000 - fp0.5: 607551.0000
 - tn0.5: 32777282.0000 - fn0.5: 870906.0000 - precision0.5: 0.8929 - recall0.5: 0.8533 - tp0.7: 4775092.0000 - fp0.7: 334512.0000 - tn0.7: 33050316.0000 - fn0.7: 1161684.0000 - precision0
.7: 0.9345 - recall0.7: 0.8043 - tp0.9: 4147063.0000 - fp0.9: 95688.0000 - tn0.9: 33289134.0000 - fn0.9: 1789713.0000 - precision0.9: 0.9774 - recall0.9: 0.6985 - accuracy: 0.9624 - auc: 0
.9732 - f1: 0.2624 - val_loss: 0.3343 - val_tp0.1: 1442029.0000 - val_fp0.1: 269201.0000 - val_tn0.1: 7978594.0000 - val_fn0.1: 140576.0000 - val_precision0.1: 0.8427 - val_recall0.1: 0.91
12 - val_tp0.3: 1407074.0000 - val_fp0.3: 174843.0000 - val_tn0.3: 8072952.0000 - val_fn0.3: 175531.0000 - val_precision0.3: 0.8895 - val_recall0.3: 0.8891 - val_tp0.5: 1303677.0000 - val_
fp0.5: 75577.0000 - val_tn0.5: 8172218.0000 - val_fn0.5: 278928.0000 - val_precision0.5: 0.9452 - val_recall0.5: 0.8238 - val_tp0.7: 1215800.0000 - val_fp0.7: 37337.0000 - val_tn0.7: 82104
58.0000 - val_fn0.7: 366805.0000 - val_precision0.7: 0.9702 - val_recall0.7: 0.7682 - val_tp0.9: 998956.0000 - val_fp0.9: 6587.0000 - val_tn0.9: 8241208.0000 - val_fn0.9: 583649.0000 - val
_precision0.9: 0.9934 - val_recall0.9: 0.6312 - val_accuracy: 0.9639 - val_auc: 0.9540 - val_f1: 0.2773
Epoch 13/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2406 - tp0.1: 5706933.0000 - fp0.1: 2415054.0000 - tn0.1: 30969764.0000 - fn0.1: 229843.0000 - precision0.1: 0.7027 - rec
all0.1: 0.9613 - tp0.3: 5471726.0000 - fp0.3: 1216004.0000 - tn0.3: 32168800.0000 - fn0.3: 465050.0000 - precision0.3: 0.8182 - recall0.3: 0.9217 - tp0.5: 5159436.0000 - fp0.5: 606629.0000
 - tn0.5: 32778208.0000 - fn0.5: 777340.0000 - precision0.5: 0.8948 - recall0.5: 0.8691 - tp0.7: 4865465.0000 - fp0.7: 318865.0000 - tn0.7: 33065960.0000 - fn0.7: 1071311.0000 - precision0
.7: 0.9385 - recall0.7: 0.8195 - tp0.9: 4237026.0000 - fp0.9: 90188.0000 - tn0.9: 33294624.0000 - fn0.9: 1699750.0000 - precision0.9: 0.9792 - recall0.9: 0.7137 - accuracy: 0.9648 - auc: 0
.9754 - f1: 0.2624 - val_loss: 0.2832 - val_tp0.1: 1508198.0000 - val_fp0.1: 679566.0000 - val_tn0.1: 7568229.0000 - val_fn0.1: 74407.0000 - val_precision0.1: 0.6894 - val_recall0.1: 0.953
0 - val_tp0.3: 1446107.0000 - val_fp0.3: 334069.0000 - val_tn0.3: 7913726.0000 - val_fn0.3: 136498.0000 - val_precision0.3: 0.8123 - val_recall0.3: 0.9138 - val_tp0.5: 1377613.0000 - val_f
p0.5: 188915.0000 - val_tn0.5: 8058880.0000 - val_fn0.5: 204992.0000 - val_precision0.5: 0.8794 - val_recall0.5: 0.8705 - val_tp0.7: 1285384.0000 - val_fp0.7: 100950.0000 - val_tn0.7: 8146
845.0000 - val_fn0.7: 297221.0000 - val_precision0.7: 0.9272 - val_recall0.7: 0.8122 - val_tp0.9: 1070502.0000 - val_fp0.9: 23679.0000 - val_tn0.9: 8224116.0000 - val_fn0.9: 512103.0000 -
val_precision0.9: 0.9784 - val_recall0.9: 0.6764 - val_accuracy: 0.9599 - val_auc: 0.9699 - val_f1: 0.2773
Epoch 14/20
480/480 [==============================] - 21s 43ms/step - loss: 0.2360 - tp0.1: 5707005.0000 - fp0.1: 2311718.0000 - tn0.1: 31073108.0000 - fn0.1: 229771.0000 - precision0.1: 0.7117 - rec
all0.1: 0.9613 - tp0.3: 5487802.0000 - fp0.3: 1178541.0000 - tn0.3: 32206284.0000 - fn0.3: 448974.0000 - precision0.3: 0.8232 - recall0.3: 0.9244 - tp0.5: 5194613.0000 - fp0.5: 583262.0000
 - tn0.5: 32801574.0000 - fn0.5: 742163.0000 - precision0.5: 0.8991 - recall0.5: 0.8750 - tp0.7: 4910863.0000 - fp0.7: 316507.0000 - tn0.7: 33068318.0000 - fn0.7: 1025913.0000 - precision0
.7: 0.9395 - recall0.7: 0.8272 - tp0.9: 4295202.0000 - fp0.9: 88374.0000 - tn0.9: 33296440.0000 - fn0.9: 1641574.0000 - precision0.9: 0.9798 - recall0.9: 0.7235 - accuracy: 0.9663 - auc: 0
.9761 - f1: 0.2624 - val_loss: 0.2611 - val_tp0.1: 1505124.0000 - val_fp0.1: 555660.0000 - val_tn0.1: 7692135.0000 - val_fn0.1: 77481.0000 - val_precision0.1: 0.7304 - val_recall0.1: 0.951
0 - val_tp0.3: 1441677.0000 - val_fp0.3: 299314.0000 - val_tn0.3: 7948481.0000 - val_fn0.3: 140928.0000 - val_precision0.3: 0.8281 - val_recall0.3: 0.9110 - val_tp0.5: 1368785.0000 - val_f
p0.5: 147030.0000 - val_tn0.5: 8100765.0000 - val_fn0.5: 213820.0000 - val_precision0.5: 0.9030 - val_recall0.5: 0.8649 - val_tp0.7: 1283194.0000 - val_fp0.7: 69148.0000 - val_tn0.7: 81786
47.0000 - val_fn0.7: 299411.0000 - val_precision0.7: 0.9489 - val_recall0.7: 0.8108 - val_tp0.9: 1099339.0000 - val_fp0.9: 14278.0000 - val_tn0.9: 8233517.0000 - val_fn0.9: 483266.0000 - v
al_precision0.9: 0.9872 - val_recall0.9: 0.6946 - val_accuracy: 0.9633 - val_auc: 0.9706 - val_f1: 0.2773
Epoch 15/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2284 - tp0.1: 5723340.0000 - fp0.1: 2332080.0000 - tn0.1: 31052740.0000 - fn0.1: 213436.0000 - precision0.1: 0.7105 - rec
all0.1: 0.9640 - tp0.3: 5500588.0000 - fp0.3: 1197957.0000 - tn0.3: 32186864.0000 - fn0.3: 436188.0000 - precision0.3: 0.8212 - recall0.3: 0.9265 - tp0.5: 5193950.0000 - fp0.5: 592030.0000
 - tn0.5: 32792800.0000 - fn0.5: 742826.0000 - precision0.5: 0.8977 - recall0.5: 0.8749 - tp0.7: 4891648.0000 - fp0.7: 303851.0000 - tn0.7: 33080976.0000 - fn0.7: 1045128.0000 - precision0
.7: 0.9415 - recall0.7: 0.8240 - tp0.9: 4289986.0000 - fp0.9: 85261.0000 - tn0.9: 33299542.0000 - fn0.9: 1646790.0000 - precision0.9: 0.9805 - recall0.9: 0.7226 - accuracy: 0.9661 - auc: 0
.9772 - f1: 0.2624 - val_loss: 0.2553 - val_tp0.1: 1513456.0000 - val_fp0.1: 572837.0000 - val_tn0.1: 7674958.0000 - val_fn0.1: 69149.0000 - val_precision0.1: 0.7254 - val_recall0.1: 0.956
3 - val_tp0.3: 1455197.0000 - val_fp0.3: 304127.0000 - val_tn0.3: 7943668.0000 - val_fn0.3: 127408.0000 - val_precision0.3: 0.8271 - val_recall0.3: 0.9195 - val_tp0.5: 1393774.0000 - val_f
p0.5: 176303.0000 - val_tn0.5: 8071492.0000 - val_fn0.5: 188831.0000 - val_precision0.5: 0.8877 - val_recall0.5: 0.8807 - val_tp0.7: 1315535.0000 - val_fp0.7: 92255.0000 - val_tn0.7: 81555
40.0000 - val_fn0.7: 267070.0000 - val_precision0.7: 0.9345 - val_recall0.7: 0.8312 - val_tp0.9: 1145098.0000 - val_fp0.9: 24287.0000 - val_tn0.9: 8223508.0000 - val_fn0.9: 437507.0000 - v
al_precision0.9: 0.9792 - val_recall0.9: 0.7236 - val_accuracy: 0.9629 - val_auc: 0.9728 - val_f1: 0.2773
Epoch 16/20
480/480 [==============================] - 21s 43ms/step - loss: 0.2187 - tp0.1: 5730057.0000 - fp0.1: 2238043.0000 - tn0.1: 31146768.0000 - fn0.1: 206719.0000 - precision0.1: 0.7191 - rec
all0.1: 0.9652 - tp0.3: 5526887.0000 - fp0.3: 1168477.0000 - tn0.3: 32216352.0000 - fn0.3: 409889.0000 - precision0.3: 0.8255 - recall0.3: 0.9310 - tp0.5: 5240624.0000 - fp0.5: 596087.0000
 - tn0.5: 32788744.0000 - fn0.5: 696152.0000 - precision0.5: 0.8979 - recall0.5: 0.8827 - tp0.7: 4938991.0000 - fp0.7: 293175.0000 - tn0.7: 33091644.0000 - fn0.7: 997785.0000 - precision0.
7: 0.9440 - recall0.7: 0.8319 - tp0.9: 4369873.0000 - fp0.9: 82397.0000 - tn0.9: 33302432.0000 - fn0.9: 1566903.0000 - precision0.9: 0.9815 - recall0.9: 0.7361 - accuracy: 0.9671 - auc: 0.
9784 - f1: 0.2624 - val_loss: 0.2767 - val_tp0.1: 1523233.0000 - val_fp0.1: 780683.0000 - val_tn0.1: 7467112.0000 - val_fn0.1: 59372.0000 - val_precision0.1: 0.6611 - val_recall0.1: 0.9625
 - val_tp0.3: 1444414.0000 - val_fp0.3: 310339.0000 - val_tn0.3: 7937456.0000 - val_fn0.3: 138191.0000 - val_precision0.3: 0.8231 - val_recall0.3: 0.9127 - val_tp0.5: 1372018.0000 - val_fp
0.5: 162131.0000 - val_tn0.5: 8085664.0000 - val_fn0.5: 210587.0000 - val_precision0.5: 0.8943 - val_recall0.5: 0.8669 - val_tp0.7: 1293148.0000 - val_fp0.7: 87275.0000 - val_tn0.7: 816052
0.0000 - val_fn0.7: 289457.0000 - val_precision0.7: 0.9368 - val_recall0.7: 0.8171 - val_tp0.9: 1123318.0000 - val_fp0.9: 21366.0000 - val_tn0.9: 8226429.0000 - val_fn0.9: 459287.0000 - va
l_precision0.9: 0.9813 - val_recall0.9: 0.7098 - val_accuracy: 0.9621 - val_auc: 0.9750 - val_f1: 0.2773
Epoch 17/20
480/480 [==============================] - 21s 43ms/step - loss: 0.2057 - tp0.1: 5743364.0000 - fp0.1: 2092169.0000 - tn0.1: 31292644.0000 - fn0.1: 193412.0000 - precision0.1: 0.7330 - rec
all0.1: 0.9674 - tp0.3: 5554900.0000 - fp0.3: 1112642.0000 - tn0.3: 32272176.0000 - fn0.3: 381876.0000 - precision0.3: 0.8331 - recall0.3: 0.9357 - tp0.5: 5313778.0000 - fp0.5: 596612.0000
 - tn0.5: 32788216.0000 - fn0.5: 622998.0000 - precision0.5: 0.8991 - recall0.5: 0.8951 - tp0.7: 4988960.0000 - fp0.7: 279744.0000 - tn0.7: 33105068.0000 - fn0.7: 947816.0000 - precision0.
7: 0.9469 - recall0.7: 0.8403 - tp0.9: 4440493.0000 - fp0.9: 78516.0000 - tn0.9: 33306316.0000 - fn0.9: 1496283.0000 - precision0.9: 0.9826 - recall0.9: 0.7480 - accuracy: 0.9690 - auc: 0.
9798 - f1: 0.2624 - val_loss: 0.2385 - val_tp0.1: 1523932.0000 - val_fp0.1: 579099.0000 - val_tn0.1: 7668696.0000 - val_fn0.1: 58673.0000 - val_precision0.1: 0.7246 - val_recall0.1: 0.9629
 - val_tp0.3: 1459850.0000 - val_fp0.3: 271056.0000 - val_tn0.3: 7976739.0000 - val_fn0.3: 122755.0000 - val_precision0.3: 0.8434 - val_recall0.3: 0.9224 - val_tp0.5: 1390400.0000 - val_fp
0.5: 143841.0000 - val_tn0.5: 8103954.0000 - val_fn0.5: 192205.0000 - val_precision0.5: 0.9062 - val_recall0.5: 0.8786 - val_tp0.7: 1305162.0000 - val_fp0.7: 75449.0000 - val_tn0.7: 817234
6.0000 - val_fn0.7: 277443.0000 - val_precision0.7: 0.9454 - val_recall0.7: 0.8247 - val_tp0.9: 1149169.0000 - val_fp0.9: 23232.0000 - val_tn0.9: 8224563.0000 - val_fn0.9: 433436.0000 - va
l_precision0.9: 0.9802 - val_recall0.9: 0.7261 - val_accuracy: 0.9658 - val_auc: 0.9764 - val_f1: 0.2773
Epoch 18/20
480/480 [==============================] - 21s 44ms/step - loss: 0.2093 - tp0.1: 5739105.0000 - fp0.1: 2145054.0000 - tn0.1: 31239784.0000 - fn0.1: 197671.0000 - precision0.1: 0.7279 - rec
all0.1: 0.9667 - tp0.3: 5545657.0000 - fp0.3: 1100602.0000 - tn0.3: 32284224.0000 - fn0.3: 391119.0000 - precision0.3: 0.8344 - recall0.3: 0.9341 - tp0.5: 5306649.0000 - fp0.5: 593735.0000
 - tn0.5: 32791100.0000 - fn0.5: 630127.0000 - precision0.5: 0.8994 - recall0.5: 0.8939 - tp0.7: 4982136.0000 - fp0.7: 287791.0000 - tn0.7: 33097036.0000 - fn0.7: 954640.0000 - precision0.
7: 0.9454 - recall0.7: 0.8392 - tp0.9: 4415070.0000 - fp0.9: 79761.0000 - tn0.9: 33305060.0000 - fn0.9: 1521706.0000 - precision0.9: 0.9823 - recall0.9: 0.7437 - accuracy: 0.9689 - auc: 0.
9796 - f1: 0.2624 - val_loss: 0.2350 - val_tp0.1: 1525598.0000 - val_fp0.1: 568429.0000 - val_tn0.1: 7679366.0000 - val_fn0.1: 57007.0000 - val_precision0.1: 0.7285 - val_recall0.1: 0.9640
 - val_tp0.3: 1474346.0000 - val_fp0.3: 295141.0000 - val_tn0.3: 7952654.0000 - val_fn0.3: 108259.0000 - val_precision0.3: 0.8332 - val_recall0.3: 0.9316 - val_tp0.5: 1424497.0000 - val_fp
0.5: 190561.0000 - val_tn0.5: 8057234.0000 - val_fn0.5: 158108.0000 - val_precision0.5: 0.8820 - val_recall0.5: 0.9001 - val_tp0.7: 1359738.0000 - val_fp0.7: 110514.0000 - val_tn0.7: 81372
81.0000 - val_fn0.7: 222867.0000 - val_precision0.7: 0.9248 - val_recall0.7: 0.8592 - val_tp0.9: 1242403.0000 - val_fp0.9: 43687.0000 - val_tn0.9: 8204108.0000 - val_fn0.9: 340202.0000 - v
al_precision0.9: 0.9660 - val_recall0.9: 0.7850 - val_accuracy: 0.9645 - val_auc: 0.9772 - val_f1: 0.2773
Epoch 19/20
480/480 [==============================] - 21s 43ms/step - loss: 0.1984 - tp0.1: 5744440.0000 - fp0.1: 1989366.0000 - tn0.1: 31395464.0000 - fn0.1: 192336.0000 - precision0.1: 0.7428 - rec
all0.1: 0.9676 - tp0.3: 5566674.0000 - fp0.3: 1040758.0000 - tn0.3: 32344072.0000 - fn0.3: 370102.0000 - precision0.3: 0.8425 - recall0.3: 0.9377 - tp0.5: 5348217.0000 - fp0.5: 577516.0000
 - tn0.5: 32807312.0000 - fn0.5: 588559.0000 - precision0.5: 0.9025 - recall0.5: 0.9009 - tp0.7: 5052821.0000 - fp0.7: 276996.0000 - tn0.7: 33107824.0000 - fn0.7: 883955.0000 - precision0.
7: 0.9480 - recall0.7: 0.8511 - tp0.9: 4492968.0000 - fp0.9: 72302.0000 - tn0.9: 33312520.0000 - fn0.9: 1443808.0000 - precision0.9: 0.9842 - recall0.9: 0.7568 - accuracy: 0.9703 - auc: 0.
9805 - f1: 0.2624 - val_loss: 0.2484 - val_tp0.1: 1501131.0000 - val_fp0.1: 393771.0000 - val_tn0.1: 7854024.0000 - val_fn0.1: 81474.0000 - val_precision0.1: 0.7922 - val_recall0.1: 0.9485
 - val_tp0.3: 1409758.0000 - val_fp0.3: 163787.0000 - val_tn0.3: 8084008.0000 - val_fn0.3: 172847.0000 - val_precision0.3: 0.8959 - val_recall0.3: 0.8908 - val_tp0.5: 1322183.0000 - val_fp
0.5: 84259.0000 - val_tn0.5: 8163536.0000 - val_fn0.5: 260422.0000 - val_precision0.5: 0.9401 - val_recall0.5: 0.8354 - val_tp0.7: 1216424.0000 - val_fp0.7: 42192.0000 - val_tn0.7: 8205603
.0000 - val_fn0.7: 366181.0000 - val_precision0.7: 0.9665 - val_recall0.7: 0.7686 - val_tp0.9: 1024424.0000 - val_fp0.9: 10587.0000 - val_tn0.9: 8237208.0000 - val_fn0.9: 558181.0000 - val
_precision0.9: 0.9898 - val_recall0.9: 0.6473 - val_accuracy: 0.9649 - val_auc: 0.9712 - val_f1: 0.2773
Epoch 20/20
480/480 [==============================] - 21s 44ms/step - loss: 0.1942 - tp0.1: 5753146.0000 - fp0.1: 1975562.0000 - tn0.1: 31409254.0000 - fn0.1: 183630.0000 - precision0.1: 0.7444 - rec
all0.1: 0.9691 - tp0.3: 5571490.0000 - fp0.3: 1010543.0000 - tn0.3: 32374284.0000 - fn0.3: 365286.0000 - precision0.3: 0.8465 - recall0.3: 0.9385 - tp0.5: 5341643.0000 - fp0.5: 549462.0000
 - tn0.5: 32835368.0000 - fn0.5: 595133.0000 - precision0.5: 0.9067 - recall0.5: 0.8998 - tp0.7: 5040559.0000 - fp0.7: 265388.0000 - tn0.7: 33119436.0000 - fn0.7: 896217.0000 - precision0.
7: 0.9500 - recall0.7: 0.8490 - tp0.9: 4493527.0000 - fp0.9: 71308.0000 - tn0.9: 33313516.0000 - fn0.9: 1443249.0000 - precision0.9: 0.9844 - recall0.9: 0.7569 - accuracy: 0.9709 - auc: 0.
9814 - f1: 0.2624 - val_loss: 0.2216 - val_tp0.1: 1527498.0000 - val_fp0.1: 536072.0000 - val_tn0.1: 7711723.0000 - val_fn0.1: 55107.0000 - val_precision0.1: 0.7402 - val_recall0.1: 0.9652
 - val_tp0.3: 1473395.0000 - val_fp0.3: 266651.0000 - val_tn0.3: 7981144.0000 - val_fn0.3: 109210.0000 - val_precision0.3: 0.8468 - val_recall0.3: 0.9310 - val_tp0.5: 1410224.0000 - val_fp
0.5: 150868.0000 - val_tn0.5: 8096927.0000 - val_fn0.5: 172381.0000 - val_precision0.5: 0.9034 - val_recall0.5: 0.8911 - val_tp0.7: 1334322.0000 - val_fp0.7: 77751.0000 - val_tn0.7: 817004
4.0000 - val_fn0.7: 248283.0000 - val_precision0.7: 0.9449 - val_recall0.7: 0.8431 - val_tp0.9: 1186212.0000 - val_fp0.9: 22796.0000 - val_tn0.9: 8224999.0000 - val_fn0.9: 396393.0000 - va
l_precision0.9: 0.9811 - val_recall0.9: 0.7495 - val_accuracy: 0.9671 - val_auc: 0.9782 - val_f1: 0.2773
--- Running training session 57/140
{'hp_epochs': 20, 'hp_batch_size': 16, 'hp_scaler': 'standard', 'hp_n_levels': 3, 'hp_first_filters': 64, 'hp_pool_size': 2, 'hp_input_size': 16384, 'hp_lr_start': 0.05209134418613575, 'hp
_lr_power': 5.0}
--- repeat #: 1
input - shape:   (None, 16384, 1)
output - shape:  (None, 16384, 1)
Epoch 1/20
300/300 [==============================] - 131s 378ms/step - loss: 1.1698 - tp0.1: 9792293.0000 - fp0.1: 16380066.0000 - tn0.1: 50293360.0000 - fn0.1: 2177474.0000 - precision0.1: 0.3741 -
 recall0.1: 0.8181 - tp0.3: 8106203.0000 - fp0.3: 5386658.0000 - tn0.3: 61286792.0000 - fn0.3: 3863564.0000 - precision0.3: 0.6008 - recall0.3: 0.6772 - tp0.5: 6816180.0000 - fp0.5: 244709
8.0000 - tn0.5: 64226336.0000 - fn0.5: 5153587.0000 - precision0.5: 0.7358 - recall0.5: 0.5694 - tp0.7: 5263785.0000 - fp0.7: 811449.0000 - tn0.7: 65861992.0000 - fn0.7: 6705982.0000 - pre
cision0.7: 0.8664 - recall0.7: 0.4398 - tp0.9: 3484091.0000 - fp0.9: 160303.0000 - tn0.9: 66513160.0000 - fn0.9: 8485676.0000 - precision0.9: 0.9560 - recall0.9: 0.2911 - accuracy: 0.9034
- auc: 0.8642 - f1: 0.2642 - val_loss: 2.1107 - val_tp0.1: 3108339.0000 - val_fp0.1: 15105390.0000 - val_tn0.1: 1392556.0000 - val_fn0.1: 54515.0000 - val_precision0.1: 0.1707 - val_recall
0.1: 0.9828 - val_tp0.3: 2913184.0000 - val_fp0.3: 8671661.0000 - val_tn0.3: 7826285.0000 - val_fn0.3: 249670.0000 - val_precision0.3: 0.2515 - val_recall0.3: 0.9211 - val_tp0.5: 2807049.0
000 - val_fp0.5: 6353778.0000 - val_tn0.5: 10144168.0000 - val_fn0.5: 355805.0000 - val_precision0.5: 0.3064 - val_recall0.5: 0.8875 - val_tp0.7: 2699435.0000 - val_fp0.7: 4581266.0000 - v
al_tn0.7: 11916680.0000 - val_fn0.7: 463419.0000 - val_precision0.7: 0.3708 - val_recall0.7: 0.8535 - val_tp0.9: 2536045.0000 - val_fp0.9: 2954857.0000 - val_tn0.9: 13543089.0000 - val_fn0
.9: 626809.0000 - val_precision0.9: 0.4619 - val_recall0.9: 0.8018 - val_accuracy: 0.6587 - val_auc: 0.8551 - val_f1: 0.2772
Epoch 2/20
300/300 [==============================] - 110s 367ms/step - loss: 1.1426 - tp0.1: 9916409.0000 - fp0.1: 13548468.0000 - tn0.1: 53124976.0000 - fn0.1: 2053358.0000 - precision0.1: 0.4226 -
 recall0.1: 0.8285 - tp0.3: 8478833.0000 - fp0.3: 4638495.0000 - tn0.3: 62034944.0000 - fn0.3: 3490934.0000 - precision0.3: 0.6464 - recall0.3: 0.7084 - tp0.5: 7332170.0000 - fp0.5: 218002
8.0000 - tn0.5: 64493416.0000 - fn0.5: 4637597.0000 - precision0.5: 0.7708 - recall0.5: 0.6126 - tp0.7: 5950349.0000 - fp0.7: 846977.0000 - tn0.7: 65826468.0000 - fn0.7: 6019418.0000 - pre
cision0.7: 0.8754 - recall0.7: 0.4971 - tp0.9: 4092626.0000 - fp0.9: 183109.0000 - tn0.9: 66490320.0000 - fn0.9: 7877141.0000 - precision0.9: 0.9572 - recall0.9: 0.3419 - accuracy: 0.9133
- auc: 0.8817 - f1: 0.2642 - val_loss: 1.3552 - val_tp0.1: 3018867.0000 - val_fp0.1: 9642342.0000 - val_tn0.1: 6855604.0000 - val_fn0.1: 143987.0000 - val_precision0.1: 0.2384 - val_recall
0.1: 0.9545 - val_tp0.3: 2882930.0000 - val_fp0.3: 5948958.0000 - val_tn0.3: 10548988.0000 - val_fn0.3: 279924.0000 - val_precision0.3: 0.3264 - val_recall0.3: 0.9115 - val_tp0.5: 2710540.
0000 - val_fp0.5: 3482144.0000 - val_tn0.5: 13015802.0000 - val_fn0.5: 452314.0000 - val_precision0.5: 0.4377 - val_recall0.5: 0.8570 - val_tp0.7: 2470093.0000 - val_fp0.7: 1883006.0000 -
val_tn0.7: 14614940.0000 - val_fn0.7: 692761.0000 - val_precision0.7: 0.5674 - val_recall0.7: 0.7810 - val_tp0.9: 2057360.0000 - val_fp0.9: 785358.0000 - val_tn0.9: 15712588.0000 - val_fn0
.9: 1105494.0000 - val_precision0.9: 0.7237 - val_recall0.9: 0.6505 - val_accuracy: 0.7999 - val_auc: 0.8997 - val_f1: 0.2772
Epoch 3/20
300/300 [==============================] - 110s 367ms/step - loss: 1.1129 - tp0.1: 10053816.0000 - fp0.1: 11055564.0000 - tn0.1: 55617880.0000 - fn0.1: 1915951.0000 - precision0.1: 0.4763
- recall0.1: 0.8399 - tp0.3: 8934470.0000 - fp0.3: 4064684.0000 - tn0.3: 62608744.0000 - fn0.3: 3035297.0000 - precision0.3: 0.6873 - recall0.3: 0.7464 - tp0.5: 8010627.0000 - fp0.5: 21114
72.0000 - tn0.5: 64561976.0000 - fn0.5: 3959140.0000 - precision0.5: 0.7914 - recall0.5: 0.6692 - tp0.7: 6744537.0000 - fp0.7: 920340.0000 - tn0.7: 65753096.0000 - fn0.7: 5225230.0000 - pr
ecision0.7: 0.8799 - recall0.7: 0.5635 - tp0.9: 4739266.0000 - fp0.9: 210718.0000 - tn0.9: 66462700.0000 - fn0.9: 7230501.0000 - precision0.9: 0.9574 - recall0.9: 0.3959 - accuracy: 0.9228
 - auc: 0.9043 - f1: 0.2642 - val_loss: 1.1474 - val_tp0.1: 2967564.0000 - val_fp0.1: 6983406.0000 - val_tn0.1: 9514540.0000 - val_fn0.1: 195290.0000 - val_precision0.1: 0.2982 - val_recal
l0.1: 0.9383 - val_tp0.3: 2788083.0000 - val_fp0.3: 2981666.0000 - val_tn0.3: 13516280.0000 - val_fn0.3: 374771.0000 - val_precision0.3: 0.4832 - val_recall0.3: 0.8815 - val_tp0.5: 2559864
.0000 - val_fp0.5: 1470225.0000 - val_tn0.5: 15027721.0000 - val_fn0.5: 602990.0000 - val_precision0.5: 0.6352 - val_recall0.5: 0.8094 - val_tp0.7: 2204980.0000 - val_fp0.7: 672610.0000 -
val_tn0.7: 15825336.0000 - val_fn0.7: 957874.0000 - val_precision0.7: 0.7663 - val_recall0.7: 0.6971 - val_tp0.9: 1441024.0000 - val_fp0.9: 161378.0000 - val_tn0.9: 16336568.0000 - val_fn0
.9: 1721830.0000 - val_precision0.9: 0.8993 - val_recall0.9: 0.4556 - val_accuracy: 0.8946 - val_auc: 0.9228 - val_f1: 0.2772
Epoch 4/20
300/300 [==============================] - 110s 367ms/step - loss: 0.9937 - tp0.1: 10107575.0000 - fp0.1: 10632681.0000 - tn0.1: 56040784.0000 - fn0.1: 1862192.0000 - precision0.1: 0.4873
- recall0.1: 0.8444 - tp0.3: 9018636.0000 - fp0.3: 3861193.0000 - tn0.3: 62812244.0000 - fn0.3: 2951131.0000 - precision0.3: 0.7002 - recall0.3: 0.7535 - tp0.5: 8145541.0000 - fp0.5: 20363
45.0000 - tn0.5: 64637076.0000 - fn0.5: 3824226.0000 - precision0.5: 0.8000 - recall0.5: 0.6805 - tp0.7: 6999441.0000 - fp0.7: 948132.0000 - tn0.7: 65725288.0000 - fn0.7: 4970326.0000 - pr
ecision0.7: 0.8807 - recall0.7: 0.5848 - tp0.9: 4926381.0000 - fp0.9: 215719.0000 - tn0.9: 66457736.0000 - fn0.9: 7043386.0000 - precision0.9: 0.9580 - recall0.9: 0.4116 - accuracy: 0.9255
 - auc: 0.9063 - f1: 0.2642 - val_loss: 0.8383 - val_tp0.1: 2770907.0000 - val_fp0.1: 3233641.0000 - val_tn0.1: 13264305.0000 - val_fn0.1: 391947.0000 - val_precision0.1: 0.4615 - val_reca
ll0.1: 0.8761 - val_tp0.3: 2695153.0000 - val_fp0.3: 2457686.0000 - val_tn0.3: 14040260.0000 - val_fn0.3: 467701.0000 - val_precision0.3: 0.5230 - val_recall0.3: 0.8521 - val_tp0.5: 262682
4.0000 - val_fp0.5: 1996722.0000 - val_tn0.5: 14501224.0000 - val_fn0.5: 536030.0000 - val_precision0.5: 0.5681 - val_recall0.5: 0.8305 - val_tp0.7: 2543169.0000 - val_fp0.7: 1571034.0000
- val_tn0.7: 14926912.0000 - val_fn0.7: 619685.0000 - val_precision0.7: 0.6181 - val_recall0.7: 0.8041 - val_tp0.9: 2364543.0000 - val_fp0.9: 988221.0000 - val_tn0.9: 15509725.0000 - val_f
n0.9: 798311.0000 - val_precision0.9: 0.7053 - val_recall0.9: 0.7476 - val_accuracy: 0.8712 - val_auc: 0.9011 - val_f1: 0.2772
Epoch 5/20
300/300 [==============================] - 110s 366ms/step - loss: 0.6515 - tp0.1: 10120253.0000 - fp0.1: 9513561.0000 - tn0.1: 57159892.0000 - fn0.1: 1849514.0000 - precision0.1: 0.5155 -
 recall0.1: 0.8455 - tp0.3: 9146172.0000 - fp0.3: 3874592.0000 - tn0.3: 62798872.0000 - fn0.3: 2823595.0000 - precision0.3: 0.7024 - recall0.3: 0.7641 - tp0.5: 8285447.0000 - fp0.5: 203234
9.0000 - tn0.5: 64641116.0000 - fn0.5: 3684320.0000 - precision0.5: 0.8030 - recall0.5: 0.6922 - tp0.7: 7115867.0000 - fp0.7: 936057.0000 - tn0.7: 65737360.0000 - fn0.7: 4853900.0000 - pre
cision0.7: 0.8837 - recall0.7: 0.5945 - tp0.9: 5021875.0000 - fp0.9: 210954.0000 - tn0.9: 66462488.0000 - fn0.9: 6947892.0000 - precision0.9: 0.9597 - recall0.9: 0.4195 - accuracy: 0.9273
- auc: 0.9005 - f1: 0.2642 - val_loss: 0.5790 - val_tp0.1: 2786753.0000 - val_fp0.1: 2338059.0000 - val_tn0.1: 14159887.0000 - val_fn0.1: 376101.0000 - val_precision0.1: 0.5438 - val_recal
l0.1: 0.8811 - val_tp0.3: 2442329.0000 - val_fp0.3: 767449.0000 - val_tn0.3: 15730497.0000 - val_fn0.3: 720525.0000 - val_precision0.3: 0.7609 - val_recall0.3: 0.7722 - val_tp0.5: 2227274.
0000 - val_fp0.5: 432048.0000 - val_tn0.5: 16065898.0000 - val_fn0.5: 935580.0000 - val_precision0.5: 0.8375 - val_recall0.5: 0.7042 - val_tp0.7: 1917679.0000 - val_fp0.7: 203533.0000 - va
l_tn0.7: 16294413.0000 - val_fn0.7: 1245175.0000 - val_precision0.7: 0.9040 - val_recall0.7: 0.6063 - val_tp0.9: 1380335.0000 - val_fp0.9: 47607.0000 - val_tn0.9: 16450339.0000 - val_fn0.9
: 1782519.0000 - val_precision0.9: 0.9667 - val_recall0.9: 0.4364 - val_accuracy: 0.9304 - val_auc: 0.9216 - val_f1: 0.2772
Epoch 6/20
300/300 [==============================] - 110s 367ms/step - loss: 0.6145 - tp0.1: 10221226.0000 - fp0.1: 9054753.0000 - tn0.1: 57618652.0000 - fn0.1: 1748541.0000 - precision0.1: 0.5303 -
 recall0.1: 0.8539 - tp0.3: 9253443.0000 - fp0.3: 3526719.0000 - tn0.3: 63146724.0000 - fn0.3: 2716324.0000 - precision0.3: 0.7240 - recall0.3: 0.7731 - tp0.5: 8544655.0000 - fp0.5: 198031
7.0000 - tn0.5: 64693120.0000 - fn0.5: 3425112.0000 - precision0.5: 0.8118 - recall0.5: 0.7139 - tp0.7: 7486882.0000 - fp0.7: 931061.0000 - tn0.7: 65742384.0000 - fn0.7: 4482885.0000 - pre
cision0.7: 0.8894 - recall0.7: 0.6255 - tp0.9: 5438772.0000 - fp0.9: 216030.0000 - tn0.9: 66457408.0000 - fn0.9: 6530995.0000 - precision0.9: 0.9618 - recall0.9: 0.4544 - accuracy: 0.9313
- auc: 0.9073 - f1: 0.2642 - val_loss: 0.5665 - val_tp0.1: 2863152.0000 - val_fp0.1: 2970240.0000 - val_tn0.1: 13527706.0000 - val_fn0.1: 299702.0000 - val_precision0.1: 0.4908 - val_recal
l0.1: 0.9052 - val_tp0.3: 2546439.0000 - val_fp0.3: 915799.0000 - val_tn0.3: 15582147.0000 - val_fn0.3: 616415.0000 - val_precision0.3: 0.7355 - val_recall0.3: 0.8051 - val_tp0.5: 2318053.
0000 - val_fp0.5: 477192.0000 - val_tn0.5: 16020754.0000 - val_fn0.5: 844801.0000 - val_precision0.5: 0.8293 - val_recall0.5: 0.7329 - val_tp0.7: 1952977.0000 - val_fp0.7: 192829.0000 - va
l_tn0.7: 16305117.0000 - val_fn0.7: 1209877.0000 - val_precision0.7: 0.9101 - val_recall0.7: 0.6175 - val_tp0.9: 1328714.0000 - val_fp0.9: 31660.0000 - val_tn0.9: 16466286.0000 - val_fn0.9
: 1834140.0000 - val_precision0.9: 0.9767 - val_recall0.9: 0.4201 - val_accuracy: 0.9328 - val_auc: 0.9303 - val_f1: 0.2772
Epoch 7/20
300/300 [==============================] - 110s 367ms/step - loss: 0.6010 - tp0.1: 10240926.0000 - fp0.1: 8765134.0000 - tn0.1: 57908304.0000 - fn0.1: 1728841.0000 - precision0.1: 0.5388 -
 recall0.1: 0.8556 - tp0.3: 9298657.0000 - fp0.3: 3414297.0000 - tn0.3: 63259112.0000 - fn0.3: 2671110.0000 - precision0.3: 0.7314 - recall0.3: 0.7768 - tp0.5: 8600118.0000 - fp0.5: 190447
1.0000 - tn0.5: 64768972.0000 - fn0.5: 3369649.0000 - precision0.5: 0.8187 - recall0.5: 0.7185 - tp0.7: 7578223.0000 - fp0.7: 896266.0000 - tn0.7: 65777152.0000 - fn0.7: 4391544.0000 - pre
cision0.7: 0.8942 - recall0.7: 0.6331 - tp0.9: 5575318.0000 - fp0.9: 206423.0000 - tn0.9: 66467012.0000 - fn0.9: 6394449.0000 - precision0.9: 0.9643 - recall0.9: 0.4658 - accuracy: 0.9329
- auc: 0.9091 - f1: 0.2642 - val_loss: 0.6136 - val_tp0.1: 2833227.0000 - val_fp0.1: 2966233.0000 - val_tn0.1: 13531713.0000 - val_fn0.1: 329627.0000 - val_precision0.1: 0.4885 - val_recal
l0.1: 0.8958 - val_tp0.3: 2657099.0000 - val_fp0.3: 1503941.0000 - val_tn0.3: 14994005.0000 - val_fn0.3: 505755.0000 - val_precision0.3: 0.6386 - val_recall0.3: 0.8401 - val_tp0.5: 2521378
.0000 - val_fp0.5: 979303.0000 - val_tn0.5: 15518643.0000 - val_fn0.5: 641476.0000 - val_precision0.5: 0.7203 - val_recall0.5: 0.7972 - val_tp0.7: 2355434.0000 - val_fp0.7: 617852.0000 - v
al_tn0.7: 15880094.0000 - val_fn0.7: 807420.0000 - val_precision0.7: 0.7922 - val_recall0.7: 0.7447 - val_tp0.9: 2044932.0000 - val_fp0.9: 285145.0000 - val_tn0.9: 16212801.0000 - val_fn0.
9: 1117922.0000 - val_precision0.9: 0.8776 - val_recall0.9: 0.6465 - val_accuracy: 0.9176 - val_auc: 0.9226 - val_f1: 0.2772
Epoch 8/20
300/300 [==============================] - 110s 367ms/step - loss: 0.5874 - tp0.1: 10313883.0000 - fp0.1: 8946511.0000 - tn0.1: 57726912.0000 - fn0.1: 1655884.0000 - precision0.1: 0.5355 -
 recall0.1: 0.8617 - tp0.3: 9372263.0000 - fp0.3: 3402993.0000 - tn0.3: 63270404.0000 - fn0.3: 2597504.0000 - precision0.3: 0.7336 - recall0.3: 0.7830 - tp0.5: 8691543.0000 - fp0.5: 189512
5.0000 - tn0.5: 64778312.0000 - fn0.5: 3278224.0000 - precision0.5: 0.8210 - recall0.5: 0.7261 - tp0.7: 7678420.0000 - fp0.7: 891412.0000 - tn0.7: 65782052.0000 - fn0.7: 4291347.0000 - pre
cision0.7: 0.8960 - recall0.7: 0.6415 - tp0.9: 5663701.0000 - fp0.9: 205730.0000 - tn0.9: 66467716.0000 - fn0.9: 6306066.0000 - precision0.9: 0.9649 - recall0.9: 0.4732 - accuracy: 0.9342
- auc: 0.9123 - f1: 0.2642 - val_loss: 0.5994 - val_tp0.1: 2655699.0000 - val_fp0.1: 1220112.0000 - val_tn0.1: 15277834.0000 - val_fn0.1: 507155.0000 - val_precision0.1: 0.6852 - val_recal
l0.1: 0.8397 - val_tp0.3: 2195697.0000 - val_fp0.3: 383330.0000 - val_tn0.3: 16114616.0000 - val_fn0.3: 967157.0000 - val_precision0.3: 0.8514 - val_recall0.3: 0.6942 - val_tp0.5: 1922446.
0000 - val_fp0.5: 203083.0000 - val_tn0.5: 16294863.0000 - val_fn0.5: 1240408.0000 - val_precision0.5: 0.9045 - val_recall0.5: 0.6078 - val_tp0.7: 1577371.0000 - val_fp0.7: 87819.0000 - va
l_tn0.7: 16410127.0000 - val_fn0.7: 1585483.0000 - val_precision0.7: 0.9473 - val_recall0.7: 0.4987 - val_tp0.9: 1033785.0000 - val_fp0.9: 15309.0000 - val_tn0.9: 16482637.0000 - val_fn0.9
: 2129069.0000 - val_precision0.9: 0.9854 - val_recall0.9: 0.3269 - val_accuracy: 0.9266 - val_auc: 0.9095 - val_f1: 0.2772
Epoch 9/20
300/300 [==============================] - 110s 366ms/step - loss: 0.5779 - tp0.1: 10317324.0000 - fp0.1: 8680261.0000 - tn0.1: 57993180.0000 - fn0.1: 1652443.0000 - precision0.1: 0.5431 -
 recall0.1: 0.8619 - tp0.3: 9391398.0000 - fp0.3: 3311151.0000 - tn0.3: 63362284.0000 - fn0.3: 2578369.0000 - precision0.3: 0.7393 - recall0.3: 0.7846 - tp0.5: 8705608.0000 - fp0.5: 181412
3.0000 - tn0.5: 64859340.0000 - fn0.5: 3264159.0000 - precision0.5: 0.8276 - recall0.5: 0.7273 - tp0.7: 7731054.0000 - fp0.7: 858059.0000 - tn0.7: 65815376.0000 - fn0.7: 4238713.0000 - pre
cision0.7: 0.9001 - recall0.7: 0.6459 - tp0.9: 5763101.0000 - fp0.9: 200933.0000 - tn0.9: 66472512.0000 - fn0.9: 6206666.0000 - precision0.9: 0.9663 - recall0.9: 0.4815 - accuracy: 0.9354
- auc: 0.9133 - f1: 0.2642 - val_loss: 0.5336 - val_tp0.1: 2783751.0000 - val_fp0.1: 1864654.0000 - val_tn0.1: 14633292.0000 - val_fn0.1: 379103.0000 - val_precision0.1: 0.5989 - val_recal
l0.1: 0.8801 - val_tp0.3: 2518451.0000 - val_fp0.3: 718668.0000 - val_tn0.3: 15779278.0000 - val_fn0.3: 644403.0000 - val_precision0.3: 0.7780 - val_recall0.3: 0.7963 - val_tp0.5: 2312604.
0000 - val_fp0.5: 409725.0000 - val_tn0.5: 16088221.0000 - val_fn0.5: 850250.0000 - val_precision0.5: 0.8495 - val_recall0.5: 0.7312 - val_tp0.7: 2020053.0000 - val_fp0.7: 204600.0000 - va
l_tn0.7: 16293346.0000 - val_fn0.7: 1142801.0000 - val_precision0.7: 0.9080 - val_recall0.7: 0.6387 - val_tp0.9: 1443951.0000 - val_fp0.9: 46830.0000 - val_tn0.9: 16451116.0000 - val_fn0.9
: 1718903.0000 - val_precision0.9: 0.9686 - val_recall0.9: 0.4565 - val_accuracy: 0.9359 - val_auc: 0.9250 - val_f1: 0.2772
Epoch 10/20
300/300 [==============================] - 110s 367ms/step - loss: 0.5662 - tp0.1: 10352395.0000 - fp0.1: 8545400.0000 - tn0.1: 58128032.0000 - fn0.1: 1617372.0000 - precision0.1: 0.5478 -
 recall0.1: 0.8649 - tp0.3: 9453462.0000 - fp0.3: 3297727.0000 - tn0.3: 63375704.0000 - fn0.3: 2516305.0000 - precision0.3: 0.7414 - recall0.3: 0.7898 - tp0.5: 8780933.0000 - fp0.5: 180115
0.0000 - tn0.5: 64872296.0000 - fn0.5: 3188834.0000 - precision0.5: 0.8298 - recall0.5: 0.7336 - tp0.7: 7844065.0000 - fp0.7: 860857.0000 - tn0.7: 65812548.0000 - fn0.7: 4125702.0000 - pre
cision0.7: 0.9011 - recall0.7: 0.6553 - tp0.9: 5892491.0000 - fp0.9: 202051.0000 - tn0.9: 66471364.0000 - fn0.9: 6077276.0000 - precision0.9: 0.9668 - recall0.9: 0.4923 - accuracy: 0.9365
- auc: 0.9156 - f1: 0.2642 - val_loss: 0.5278 - val_tp0.1: 2876358.0000 - val_fp0.1: 2715037.0000 - val_tn0.1: 13782909.0000 - val_fn0.1: 286496.0000 - val_precision0.1: 0.5144 - val_recal
l0.1: 0.9094 - val_tp0.3: 2636089.0000 - val_fp0.3: 1014420.0000 - val_tn0.3: 15483526.0000 - val_fn0.3: 526765.0000 - val_precision0.3: 0.7221 - val_recall0.3: 0.8335 - val_tp0.5: 2435712
.0000 - val_fp0.5: 548278.0000 - val_tn0.5: 15949668.0000 - val_fn0.5: 727142.0000 - val_precision0.5: 0.8163 - val_recall0.5: 0.7701 - val_tp0.7: 2160433.0000 - val_fp0.7: 261165.0000 - v
al_tn0.7: 16236781.0000 - val_fn0.7: 1002421.0000 - val_precision0.7: 0.8922 - val_recall0.7: 0.6831 - val_tp0.9: 1556420.0000 - val_fp0.9: 50492.0000 - val_tn0.9: 16447454.0000 - val_fn0.
9: 1606434.0000 - val_precision0.9: 0.9686 - val_recall0.9: 0.4921 - val_accuracy: 0.9351 - val_auc: 0.9348 - val_f1: 0.2772
Epoch 11/20
300/300 [==============================] - 110s 367ms/step - loss: 0.5610 - tp0.1: 10373608.0000 - fp0.1: 8572388.0000 - tn0.1: 58101040.0000 - fn0.1: 1596159.0000 - precision0.1: 0.5475 -
 recall0.1: 0.8667 - tp0.3: 9468020.0000 - fp0.3: 3221384.0000 - tn0.3: 63452048.0000 - fn0.3: 2501747.0000 - precision0.3: 0.7461 - recall0.3: 0.7910 - tp0.5: 8789688.0000 - fp0.5: 173793
5.0000 - tn0.5: 64935508.0000 - fn0.5: 3180079.0000 - precision0.5: 0.8349 - recall0.5: 0.7343 - tp0.7: 7882409.0000 - fp0.7: 841710.0000 - tn0.7: 65831744.0000 - fn0.7: 4087358.0000 - pre
cision0.7: 0.9035 - recall0.7: 0.6585 - tp0.9: 5966113.0000 - fp0.9: 196483.0000 - tn0.9: 66476952.0000 - fn0.9: 6003654.0000 - precision0.9: 0.9681 - recall0.9: 0.4984 - accuracy: 0.9375
- auc: 0.9166 - f1: 0.2642 - val_loss: 0.5227 - val_tp0.1: 2833580.0000 - val_fp0.1: 2220007.0000 - val_tn0.1: 14277939.0000 - val_fn0.1: 329274.0000 - val_precision0.1: 0.5607 - val_recal
l0.1: 0.8959 - val_tp0.3: 2545954.0000 - val_fp0.3: 744480.0000 - val_tn0.3: 15753466.0000 - val_fn0.3: 616900.0000 - val_precision0.3: 0.7737 - val_recall0.3: 0.8050 - val_tp0.5: 2303452.
0000 - val_fp0.5: 376141.0000 - val_tn0.5: 16121805.0000 - val_fn0.5: 859402.0000 - val_precision0.5: 0.8596 - val_recall0.5: 0.7283 - val_tp0.7: 1971041.0000 - val_fp0.7: 160456.0000 - va
l_tn0.7: 16337490.0000 - val_fn0.7: 1191813.0000 - val_precision0.7: 0.9247 - val_recall0.7: 0.6232 - val_tp0.9: 1375228.0000 - val_fp0.9: 28055.0000 - val_tn0.9: 16469891.0000 - val_fn0.9
: 1787626.0000 - val_precision0.9: 0.9800 - val_recall0.9: 0.4348 - val_accuracy: 0.9372 - val_auc: 0.9312 - val_f1: 0.2772
Epoch 12/20
300/300 [==============================] - 110s 366ms/step - loss: 0.5516 - tp0.1: 10421559.0000 - fp0.1: 8658554.0000 - tn0.1: 58014880.0000 - fn0.1: 1548208.0000 - precision0.1: 0.5462 -
 recall0.1: 0.8707 - tp0.3: 9509353.0000 - fp0.3: 3240736.0000 - tn0.3: 63432728.0000 - fn0.3: 2460414.0000 - precision0.3: 0.7458 - recall0.3: 0.7944 - tp0.5: 8835728.0000 - fp0.5: 174437
1.0000 - tn0.5: 64929060.0000 - fn0.5: 3134039.0000 - precision0.5: 0.8351 - recall0.5: 0.7382 - tp0.7: 7935785.0000 - fp0.7: 836444.0000 - tn0.7: 65836992.0000 - fn0.7: 4033982.0000 - pre
cision0.7: 0.9046 - recall0.7: 0.6630 - tp0.9: 6021972.0000 - fp0.9: 194282.0000 - tn0.9: 66479164.0000 - fn0.9: 5947795.0000 - precision0.9: 0.9687 - recall0.9: 0.5031 - accuracy: 0.9380
- auc: 0.9190 - f1: 0.2642 - val_loss: 0.5259 - val_tp0.1: 2813028.0000 - val_fp0.1: 2078752.0000 - val_tn0.1: 14419194.0000 - val_fn0.1: 349826.0000 - val_precision0.1: 0.5751 - val_recal
l0.1: 0.8894 - val_tp0.3: 2528314.0000 - val_fp0.3: 706366.0000 - val_tn0.3: 15791580.0000 - val_fn0.3: 634540.0000 - val_precision0.3: 0.7816 - val_recall0.3: 0.7994 - val_tp0.5: 2316024.
0000 - val_fp0.5: 388662.0000 - val_tn0.5: 16109284.0000 - val_fn0.5: 846830.0000 - val_precision0.5: 0.8563 - val_recall0.5: 0.7323 - val_tp0.7: 2044271.0000 - val_fp0.7: 193248.0000 - va
l_tn0.7: 16304698.0000 - val_fn0.7: 1118583.0000 - val_precision0.7: 0.9136 - val_recall0.7: 0.6463 - val_tp0.9: 1538039.0000 - val_fp0.9: 48557.0000 - val_tn0.9: 16449389.0000 - val_fn0.9
: 1624815.0000 - val_precision0.9: 0.9694 - val_recall0.9: 0.4863 - val_accuracy: 0.9372 - val_auc: 0.9290 - val_f1: 0.2772
Epoch 13/20
300/300 [==============================] - 110s 366ms/step - loss: 0.5454 - tp0.1: 10439462.0000 - fp0.1: 8578643.0000 - tn0.1: 58094796.0000 - fn0.1: 1530305.0000 - precision0.1: 0.5489 -
 recall0.1: 0.8722 - tp0.3: 9511162.0000 - fp0.3: 3179906.0000 - tn0.3: 63493568.0000 - fn0.3: 2458605.0000 - precision0.3: 0.7494 - recall0.3: 0.7946 - tp0.5: 8819389.0000 - fp0.5: 168064
2.0000 - tn0.5: 64992792.0000 - fn0.5: 3150378.0000 - precision0.5: 0.8399 - recall0.5: 0.7368 - tp0.7: 7930729.0000 - fp0.7: 810316.0000 - tn0.7: 65863144.0000 - fn0.7: 4039038.0000 - pre
cision0.7: 0.9073 - recall0.7: 0.6626 - tp0.9: 6037837.0000 - fp0.9: 190230.0000 - tn0.9: 66483196.0000 - fn0.9: 5931930.0000 - precision0.9: 0.9695 - recall0.9: 0.5044 - accuracy: 0.9386
- auc: 0.9201 - f1: 0.2642 - val_loss: 0.5152 - val_tp0.1: 2825402.0000 - val_fp0.1: 2086657.0000 - val_tn0.1: 14411289.0000 - val_fn0.1: 337452.0000 - val_precision0.1: 0.5752 - val_recal
l0.1: 0.8933 - val_tp0.3: 2573708.0000 - val_fp0.3: 777952.0000 - val_tn0.3: 15719994.0000 - val_fn0.3: 589146.0000 - val_precision0.3: 0.7679 - val_recall0.3: 0.8137 - val_tp0.5: 2360881.
0000 - val_fp0.5: 416427.0000 - val_tn0.5: 16081519.0000 - val_fn0.5: 801973.0000 - val_precision0.5: 0.8501 - val_recall0.5: 0.7464 - val_tp0.7: 2074036.0000 - val_fp0.7: 196402.0000 - va
l_tn0.7: 16301544.0000 - val_fn0.7: 1088818.0000 - val_precision0.7: 0.9135 - val_recall0.7: 0.6557 - val_tp0.9: 1511886.0000 - val_fp0.9: 41512.0000 - val_tn0.9: 16456434.0000 - val_fn0.9
: 1650968.0000 - val_precision0.9: 0.9733 - val_recall0.9: 0.4780 - val_accuracy: 0.9380 - val_auc: 0.9310 - val_f1: 0.2772
Epoch 14/20
300/300 [==============================] - 110s 366ms/step - loss: 0.5443 - tp0.1: 10444511.0000 - fp0.1: 8600961.0000 - tn0.1: 58072476.0000 - fn0.1: 1525256.0000 - precision0.1: 0.5484 -
 recall0.1: 0.8726 - tp0.3: 9538913.0000 - fp0.3: 3252925.0000 - tn0.3: 63420496.0000 - fn0.3: 2430854.0000 - precision0.3: 0.7457 - recall0.3: 0.7969 - tp0.5: 8849106.0000 - fp0.5: 171047
5.0000 - tn0.5: 64962976.0000 - fn0.5: 3120661.0000 - precision0.5: 0.8380 - recall0.5: 0.7393 - tp0.7: 7973420.0000 - fp0.7: 827235.0000 - tn0.7: 65846200.0000 - fn0.7: 3996347.0000 - pre
cision0.7: 0.9060 - recall0.7: 0.6661 - tp0.9: 6091812.0000 - fp0.9: 195928.0000 - tn0.9: 66477488.0000 - fn0.9: 5877955.0000 - precision0.9: 0.9688 - recall0.9: 0.5089 - accuracy: 0.9386
- auc: 0.9203 - f1: 0.2642 - val_loss: 0.5191 - val_tp0.1: 2826998.0000 - val_fp0.1: 2155571.0000 - val_tn0.1: 14342375.0000 - val_fn0.1: 335856.0000 - val_precision0.1: 0.5674 - val_recal
l0.1: 0.8938 - val_tp0.3: 2586837.0000 - val_fp0.3: 829620.0000 - val_tn0.3: 15668326.0000 - val_fn0.3: 576017.0000 - val_precision0.3: 0.7572 - val_recall0.3: 0.8179 - val_tp0.5: 2383398.
0000 - val_fp0.5: 453397.0000 - val_tn0.5: 16044549.0000 - val_fn0.5: 779456.0000 - val_precision0.5: 0.8402 - val_recall0.5: 0.7536 - val_tp0.7: 2122357.0000 - val_fp0.7: 227187.0000 - va
l_tn0.7: 16270759.0000 - val_fn0.7: 1040497.0000 - val_precision0.7: 0.9033 - val_recall0.7: 0.6710 - val_tp0.9: 1608918.0000 - val_fp0.9: 56008.0000 - val_tn0.9: 16441938.0000 - val_fn0.9
: 1553936.0000 - val_precision0.9: 0.9664 - val_recall0.9: 0.5087 - val_accuracy: 0.9373 - val_auc: 0.9305 - val_f1: 0.2772
Epoch 15/20
300/300 [==============================] - 110s 367ms/step - loss: 0.5447 - tp0.1: 10433601.0000 - fp0.1: 8522557.0000 - tn0.1: 58150872.0000 - fn0.1: 1536166.0000 - precision0.1: 0.5504 -
 recall0.1: 0.8717 - tp0.3: 9526040.0000 - fp0.3: 3191360.0000 - tn0.3: 63482064.0000 - fn0.3: 2443727.0000 - precision0.3: 0.7491 - recall0.3: 0.7958 - tp0.5: 8837031.0000 - fp0.5: 167970
9.0000 - tn0.5: 64993724.0000 - fn0.5: 3132736.0000 - precision0.5: 0.8403 - recall0.5: 0.7383 - tp0.7: 7952386.0000 - fp0.7: 812351.0000 - tn0.7: 65861096.0000 - fn0.7: 4017381.0000 - pre
cision0.7: 0.9073 - recall0.7: 0.6644 - tp0.9: 6066606.0000 - fp0.9: 190872.0000 - tn0.9: 66482528.0000 - fn0.9: 5903161.0000 - precision0.9: 0.9695 - recall0.9: 0.5068 - accuracy: 0.9388
- auc: 0.9200 - f1: 0.2642 - val_loss: 0.5173 - val_tp0.1: 2837321.0000 - val_fp0.1: 2234345.0000 - val_tn0.1: 14263601.0000 - val_fn0.1: 325533.0000 - val_precision0.1: 0.5594 - val_recal
l0.1: 0.8971 - val_tp0.3: 2592118.0000 - val_fp0.3: 832258.0000 - val_tn0.3: 15665688.0000 - val_fn0.3: 570736.0000 - val_precision0.3: 0.7570 - val_recall0.3: 0.8196 - val_tp0.5: 2384808.
0000 - val_fp0.5: 447800.0000 - val_tn0.5: 16050146.0000 - val_fn0.5: 778046.0000 - val_precision0.5: 0.8419 - val_recall0.5: 0.7540 - val_tp0.7: 2112938.0000 - val_fp0.7: 217396.0000 - va
l_tn0.7: 16280550.0000 - val_fn0.7: 1049916.0000 - val_precision0.7: 0.9067 - val_recall0.7: 0.6680 - val_tp0.9: 1562455.0000 - val_fp0.9: 48138.0000 - val_tn0.9: 16449808.0000 - val_fn0.9
: 1600399.0000 - val_precision0.9: 0.9701 - val_recall0.9: 0.4940 - val_accuracy: 0.9377 - val_auc: 0.9318 - val_f1: 0.2772
Epoch 16/20
300/300 [==============================] - 110s 367ms/step - loss: 0.5470 - tp0.1: 10446770.0000 - fp0.1: 8763433.0000 - tn0.1: 57909992.0000 - fn0.1: 1522997.0000 - precision0.1: 0.5438 -
 recall0.1: 0.8728 - tp0.3: 9531541.0000 - fp0.3: 3286354.0000 - tn0.3: 63387068.0000 - fn0.3: 2438226.0000 - precision0.3: 0.7436 - recall0.3: 0.7963 - tp0.5: 8849773.0000 - fp0.5: 174540
6.0000 - tn0.5: 64928032.0000 - fn0.5: 3119994.0000 - precision0.5: 0.8353 - recall0.5: 0.7393 - tp0.7: 7984098.0000 - fp0.7: 851547.0000 - tn0.7: 65821888.0000 - fn0.7: 3985669.0000 - pre
cision0.7: 0.9036 - recall0.7: 0.6670 - tp0.9: 6119565.0000 - fp0.9: 198791.0000 - tn0.9: 66474608.0000 - fn0.9: 5850202.0000 - precision0.9: 0.9685 - recall0.9: 0.5113 - accuracy: 0.9381
- auc: 0.9200 - f1: 0.2642 - val_loss: 0.5199 - val_tp0.1: 2830415.0000 - val_fp0.1: 2198213.0000 - val_tn0.1: 14299733.0000 - val_fn0.1: 332439.0000 - val_precision0.1: 0.5629 - val_recal
l0.1: 0.8949 - val_tp0.3: 2585794.0000 - val_fp0.3: 826003.0000 - val_tn0.3: 15671943.0000 - val_fn0.3: 577060.0000 - val_precision0.3: 0.7579 - val_recall0.3: 0.8176 - val_tp0.5: 2380750.
0000 - val_fp0.5: 448412.0000 - val_tn0.5: 16049534.0000 - val_fn0.5: 782104.0000 - val_precision0.5: 0.8415 - val_recall0.5: 0.7527 - val_tp0.7: 2117268.0000 - val_fp0.7: 222683.0000 - va
l_tn0.7: 16275263.0000 - val_fn0.7: 1045586.0000 - val_precision0.7: 0.9048 - val_recall0.7: 0.6694 - val_tp0.9: 1600774.0000 - val_fp0.9: 54219.0000 - val_tn0.9: 16443727.0000 - val_fn0.9
: 1562080.0000 - val_precision0.9: 0.9672 - val_recall0.9: 0.5061 - val_accuracy: 0.9374 - val_auc: 0.9309 - val_f1: 0.2772
Epoch 17/20
300/300 [==============================] - 110s 366ms/step - loss: 0.5479 - tp0.1: 10435195.0000 - fp0.1: 8675069.0000 - tn0.1: 57998372.0000 - fn0.1: 1534572.0000 - precision0.1: 0.5461 -
 recall0.1: 0.8718 - tp0.3: 9510450.0000 - fp0.3: 3192019.0000 - tn0.3: 63481376.0000 - fn0.3: 2459317.0000 - precision0.3: 0.7487 - recall0.3: 0.7945 - tp0.5: 8815376.0000 - fp0.5: 168445
9.0000 - tn0.5: 64988968.0000 - fn0.5: 3154391.0000 - precision0.5: 0.8396 - recall0.5: 0.7365 - tp0.7: 7941450.0000 - fp0.7: 818142.0000 - tn0.7: 65855296.0000 - fn0.7: 4028317.0000 - pre
cision0.7: 0.9066 - recall0.7: 0.6635 - tp0.9: 6071147.0000 - fp0.9: 191455.0000 - tn0.9: 66481988.0000 - fn0.9: 5898620.0000 - precision0.9: 0.9694 - recall0.9: 0.5072 - accuracy: 0.9385
- auc: 0.9197 - f1: 0.2642 - val_loss: 0.5189 - val_tp0.1: 2828825.0000 - val_fp0.1: 2176033.0000 - val_tn0.1: 14321913.0000 - val_fn0.1: 334029.0000 - val_precision0.1: 0.5652 - val_recal
l0.1: 0.8944 - val_tp0.3: 2585975.0000 - val_fp0.3: 823670.0000 - val_tn0.3: 15674276.0000 - val_fn0.3: 576879.0000 - val_precision0.3: 0.7584 - val_recall0.3: 0.8176 - val_tp0.5: 2383653.
0000 - val_fp0.5: 450199.0000 - val_tn0.5: 16047747.0000 - val_fn0.5: 779201.0000 - val_precision0.5: 0.8411 - val_recall0.5: 0.7536 - val_tp0.7: 2121977.0000 - val_fp0.7: 224378.0000 - va
l_tn0.7: 16273568.0000 - val_fn0.7: 1040877.0000 - val_precision0.7: 0.9044 - val_recall0.7: 0.6709 - val_tp0.9: 1604835.0000 - val_fp0.9: 54571.0000 - val_tn0.9: 16443375.0000 - val_fn0.9
: 1558019.0000 - val_precision0.9: 0.9671 - val_recall0.9: 0.5074 - val_accuracy: 0.9375 - val_auc: 0.9309 - val_f1: 0.2772
Epoch 18/20
300/300 [==============================] - 110s 367ms/step - loss: 0.5454 - tp0.1: 10442547.0000 - fp0.1: 8683315.0000 - tn0.1: 57990116.0000 - fn0.1: 1527220.0000 - precision0.1: 0.5460 -
 recall0.1: 0.8724 - tp0.3: 9524521.0000 - fp0.3: 3195203.0000 - tn0.3: 63478232.0000 - fn0.3: 2445246.0000 - precision0.3: 0.7488 - recall0.3: 0.7957 - tp0.5: 8838166.0000 - fp0.5: 167403
0.0000 - tn0.5: 64999392.0000 - fn0.5: 3131601.0000 - precision0.5: 0.8408 - recall0.5: 0.7384 - tp0.7: 7963121.0000 - fp0.7: 806164.0000 - tn0.7: 65867296.0000 - fn0.7: 4006646.0000 - pre
cision0.7: 0.9081 - recall0.7: 0.6653 - tp0.9: 6093558.0000 - fp0.9: 189519.0000 - tn0.9: 66483896.0000 - fn0.9: 5876209.0000 - precision0.9: 0.9698 - recall0.9: 0.5091 - accuracy: 0.9389
- auc: 0.9202 - f1: 0.2642 - val_loss: 0.5184 - val_tp0.1: 2832372.0000 - val_fp0.1: 2204134.0000 - val_tn0.1: 14293812.0000 - val_fn0.1: 330482.0000 - val_precision0.1: 0.5624 - val_recal
l0.1: 0.8955 - val_tp0.3: 2594182.0000 - val_fp0.3: 844891.0000 - val_tn0.3: 15653055.0000 - val_fn0.3: 568672.0000 - val_precision0.3: 0.7543 - val_recall0.3: 0.8202 - val_tp0.5: 2395380.
0000 - val_fp0.5: 464502.0000 - val_tn0.5: 16033444.0000 - val_fn0.5: 767474.0000 - val_precision0.5: 0.8376 - val_recall0.5: 0.7573 - val_tp0.7: 2138310.0000 - val_fp0.7: 234231.0000 - va
l_tn0.7: 16263715.0000 - val_fn0.7: 1024544.0000 - val_precision0.7: 0.9013 - val_recall0.7: 0.6761 - val_tp0.9: 1623105.0000 - val_fp0.9: 57407.0000 - val_tn0.9: 16440539.0000 - val_fn0.9
: 1539749.0000 - val_precision0.9: 0.9658 - val_recall0.9: 0.5132 - val_accuracy: 0.9373 - val_auc: 0.9313 - val_f1: 0.2772
Epoch 19/20
300/300 [==============================] - 110s 366ms/step - loss: 0.5418 - tp0.1: 10456777.0000 - fp0.1: 8641725.0000 - tn0.1: 58031732.0000 - fn0.1: 1512990.0000 - precision0.1: 0.5475 -
 recall0.1: 0.8736 - tp0.3: 9535623.0000 - fp0.3: 3181102.0000 - tn0.3: 63492344.0000 - fn0.3: 2434144.0000 - precision0.3: 0.7498 - recall0.3: 0.7966 - tp0.5: 8844625.0000 - fp0.5: 167217
5.0000 - tn0.5: 65001264.0000 - fn0.5: 3125142.0000 - precision0.5: 0.8410 - recall0.5: 0.7389 - tp0.7: 7965635.0000 - fp0.7: 804103.0000 - tn0.7: 65869328.0000 - fn0.7: 4004132.0000 - pre
cision0.7: 0.9083 - recall0.7: 0.6655 - tp0.9: 6091897.0000 - fp0.9: 189993.0000 - tn0.9: 66483440.0000 - fn0.9: 5877870.0000 - precision0.9: 0.9698 - recall0.9: 0.5089 - accuracy: 0.9390
- auc: 0.9210 - f1: 0.2642 - val_loss: 0.5190 - val_tp0.1: 2831559.0000 - val_fp0.1: 2201859.0000 - val_tn0.1: 14296087.0000 - val_fn0.1: 331295.0000 - val_precision0.1: 0.5626 - val_recal
l0.1: 0.8953 - val_tp0.3: 2592759.0000 - val_fp0.3: 841402.0000 - val_tn0.3: 15656544.0000 - val_fn0.3: 570095.0000 - val_precision0.3: 0.7550 - val_recall0.3: 0.8198 - val_tp0.5: 2393128.
0000 - val_fp0.5: 461852.0000 - val_tn0.5: 16036094.0000 - val_fn0.5: 769726.0000 - val_precision0.5: 0.8382 - val_recall0.5: 0.7566 - val_tp0.7: 2135701.0000 - val_fp0.7: 232684.0000 - va
l_tn0.7: 16265262.0000 - val_fn0.7: 1027153.0000 - val_precision0.7: 0.9018 - val_recall0.7: 0.6752 - val_tp0.9: 1622963.0000 - val_fp0.9: 57421.0000 - val_tn0.9: 16440525.0000 - val_fn0.9
: 1539891.0000 - val_precision0.9: 0.9658 - val_recall0.9: 0.5131 - val_accuracy: 0.9374 - val_auc: 0.9311 - val_f1: 0.2772
Epoch 20/20
300/300 [==============================] - 110s 366ms/step - loss: 0.5454 - tp0.1: 10445310.0000 - fp0.1: 8664704.0000 - tn0.1: 58008732.0000 - fn0.1: 1524457.0000 - precision0.1: 0.5466 -
 recall0.1: 0.8726 - tp0.3: 9522539.0000 - fp0.3: 3185766.0000 - tn0.3: 63487664.0000 - fn0.3: 2447228.0000 - precision0.3: 0.7493 - recall0.3: 0.7955 - tp0.5: 8829214.0000 - fp0.5: 167729
6.0000 - tn0.5: 64996140.0000 - fn0.5: 3140553.0000 - precision0.5: 0.8404 - recall0.5: 0.7376 - tp0.7: 7945247.0000 - fp0.7: 812426.0000 - tn0.7: 65861008.0000 - fn0.7: 4024520.0000 - pre
cision0.7: 0.9072 - recall0.7: 0.6638 - tp0.9: 6065229.0000 - fp0.9: 190972.0000 - tn0.9: 66482444.0000 - fn0.9: 5904538.0000 - precision0.9: 0.9695 - recall0.9: 0.5067 - accuracy: 0.9387
- auc: 0.9203 - f1: 0.2642 - val_loss: 0.5199 - val_tp0.1: 2827480.0000 - val_fp0.1: 2173312.0000 - val_tn0.1: 14324634.0000 - val_fn0.1: 335374.0000 - val_precision0.1: 0.5654 - val_recal
l0.1: 0.8940 - val_tp0.3: 2584623.0000 - val_fp0.3: 824555.0000 - val_tn0.3: 15673391.0000 - val_fn0.3: 578231.0000 - val_precision0.3: 0.7581 - val_recall0.3: 0.8172 - val_tp0.5: 2381163.
0000 - val_fp0.5: 449934.0000 - val_tn0.5: 16048012.0000 - val_fn0.5: 781691.0000 - val_precision0.5: 0.8411 - val_recall0.5: 0.7529 - val_tp0.7: 2119470.0000 - val_fp0.7: 224462.0000 - va
l_tn0.7: 16273484.0000 - val_fn0.7: 1043384.0000 - val_precision0.7: 0.9042 - val_recall0.7: 0.6701 - val_tp0.9: 1605312.0000 - val_fp0.9: 55121.0000 - val_tn0.9: 16442825.0000 - val_fn0.9
: 1557542.0000 - val_precision0.9: 0.9668 - val_recall0.9: 0.5076 - val_accuracy: 0.9374 - val_auc: 0.9306 - val_f1: 0.2772
--- Running training session 58/140
{'hp_epochs': 20, 'hp_batch_size': 16, 'hp_scaler': 'standard', 'hp_n_levels': 3, 'hp_first_filters': 64, 'hp_pool_size': 2, 'hp_input_size': 16384, 'hp_lr_start': 0.05209134418613575, 'hp
_lr_power': 5.0}
--- repeat #: 2
input - shape:   (None, 16384, 1)
output - shape:  (None, 16384, 1)
Epoch 1/20
300/300 [==============================] - 129s 378ms/step - loss: 1.1658 - tp0.1: 9830604.0000 - fp0.1: 16232549.0000 - tn0.1: 50440888.0000 - fn0.1: 2139163.0000 - precision0.1: 0.3772 -
 recall0.1: 0.8213 - tp0.3: 8228708.0000 - fp0.3: 5302541.0000 - tn0.3: 61370904.0000 - fn0.3: 3741059.0000 - precision0.3: 0.6081 - recall0.3: 0.6875 - tp0.5: 6775084.0000 - fp0.5: 211648
8.0000 - tn0.5: 64556952.0000 - fn0.5: 5194683.0000 - precision0.5: 0.7620 - recall0.5: 0.5660 - tp0.7: 5350792.0000 - fp0.7: 805898.0000 - tn0.7: 65867540.0000 - fn0.7: 6618975.0000 - pre
cision0.7: 0.8691 - recall0.7: 0.4470 - tp0.9: 3697858.0000 - fp0.9: 194591.0000 - tn0.9: 66478832.0000 - fn0.9: 8271909.0000 - precision0.9: 0.9500 - recall0.9: 0.3089 - accuracy: 0.9070
- auc: 0.8672 - f1: 0.2642 - val_loss: 1.3447 - val_tp0.1: 3029789.0000 - val_fp0.1: 10720478.0000 - val_tn0.1: 5777468.0000 - val_fn0.1: 133065.0000 - val_precision0.1: 0.2203 - val_recal
l0.1: 0.9579 - val_tp0.3: 2725924.0000 - val_fp0.3: 5062932.0000 - val_tn0.3: 11435014.0000 - val_fn0.3: 436930.0000 - val_precision0.3: 0.3500 - val_recall0.3: 0.8619 - val_tp0.5: 2519324
.0000 - val_fp0.5: 3056247.0000 - val_tn0.5: 13441699.0000 - val_fn0.5: 643530.0000 - val_precision0.5: 0.4519 - val_recall0.5: 0.7965 - val_tp0.7: 2295185.0000 - val_fp0.7: 1804933.0000 -
 val_tn0.7: 14693013.0000 - val_fn0.7: 867669.0000 - val_precision0.7: 0.5598 - val_recall0.7: 0.7257 - val_tp0.9: 1906967.0000 - val_fp0.9: 746282.0000 - val_tn0.9: 15751664.0000 - val_fn
0.9: 1255887.0000 - val_precision0.9: 0.7187 - val_recall0.9: 0.6029 - val_accuracy: 0.8118 - val_auc: 0.8811 - val_f1: 0.2772
Epoch 2/20
300/300 [==============================] - 109s 364ms/step - loss: 1.1355 - tp0.1: 9957479.0000 - fp0.1: 13357487.0000 - tn0.1: 53315960.0000 - fn0.1: 2012288.0000 - precision0.1: 0.4271 -
 recall0.1: 0.8319 - tp0.3: 8569227.0000 - fp0.3: 4466847.0000 - tn0.3: 62206584.0000 - fn0.3: 3400540.0000 - precision0.3: 0.6573 - recall0.3: 0.7159 - tp0.5: 7483729.0000 - fp0.5: 216675
6.0000 - tn0.5: 64506688.0000 - fn0.5: 4486038.0000 - precision0.5: 0.7755 - recall0.5: 0.6252 - tp0.7: 6133208.0000 - fp0.7: 879463.0000 - tn0.7: 65793980.0000 - fn0.7: 5836559.0000 - pre
cision0.7: 0.8746 - recall0.7: 0.5124 - tp0.9: 4221172.0000 - fp0.9: 179821.0000 - tn0.9: 66493600.0000 - fn0.9: 7748595.0000 - precision0.9: 0.9591 - recall0.9: 0.3527 - accuracy: 0.9154
- auc: 0.8904 - f1: 0.2642 - val_loss: 1.1731 - val_tp0.1: 3054894.0000 - val_fp0.1: 9937096.0000 - val_tn0.1: 6560850.0000 - val_fn0.1: 107960.0000 - val_precision0.1: 0.2351 - val_recall
0.1: 0.9659 - val_tp0.3: 2783981.0000 - val_fp0.3: 3337977.0000 - val_tn0.3: 13159969.0000 - val_fn0.3: 378873.0000 - val_precision0.3: 0.4548 - val_recall0.3: 0.8802 - val_tp0.5: 2388364.
0000 - val_fp0.5: 1174330.0000 - val_tn0.5: 15323616.0000 - val_fn0.5: 774490.0000 - val_precision0.5: 0.6704 - val_recall0.5: 0.7551 - val_tp0.7: 1964516.0000 - val_fp0.7: 504808.0000 - v
al_tn0.7: 15993138.0000 - val_fn0.7: 1198338.0000 - val_precision0.7: 0.7956 - val_recall0.7: 0.6211 - val_tp0.9: 1396619.0000 - val_fp0.9: 139367.0000 - val_tn0.9: 16358579.0000 - val_fn0
.9: 1766235.0000 - val_precision0.9: 0.9093 - val_recall0.9: 0.4416 - val_accuracy: 0.9009 - val_auc: 0.9170 - val_f1: 0.2772
Epoch 3/20
300/300 [==============================] - 109s 365ms/step - loss: 1.1191 - tp0.1: 9992425.0000 - fp0.1: 11721660.0000 - tn0.1: 54951760.0000 - fn0.1: 1977342.0000 - precision0.1: 0.4602 -
 recall0.1: 0.8348 - tp0.3: 8827502.0000 - fp0.3: 4156590.0000 - tn0.3: 62516832.0000 - fn0.3: 3142265.0000 - precision0.3: 0.6799 - recall0.3: 0.7375 - tp0.5: 7896775.0000 - fp0.5: 216438
4.0000 - tn0.5: 64509016.0000 - fn0.5: 4072992.0000 - precision0.5: 0.7849 - recall0.5: 0.6597 - tp0.7: 6592308.0000 - fp0.7: 940147.0000 - tn0.7: 65733296.0000 - fn0.7: 5377459.0000 - pre
cision0.7: 0.8752 - recall0.7: 0.5507 - tp0.9: 4505055.0000 - fp0.9: 193708.0000 - tn0.9: 66479780.0000 - fn0.9: 7464712.0000 - precision0.9: 0.9588 - recall0.9: 0.3764 - accuracy: 0.9207
- auc: 0.9016 - f1: 0.2642 - val_loss: 1.1653 - val_tp0.1: 2999133.0000 - val_fp0.1: 7651011.0000 - val_tn0.1: 8846935.0000 - val_fn0.1: 163721.0000 - val_precision0.1: 0.2816 - val_recall
0.1: 0.9482 - val_tp0.3: 2753881.0000 - val_fp0.3: 2809322.0000 - val_tn0.3: 13688624.0000 - val_fn0.3: 408973.0000 - val_precision0.3: 0.4950 - val_recall0.3: 0.8707 - val_tp0.5: 2575607.
0000 - val_fp0.5: 1604092.0000 - val_tn0.5: 14893854.0000 - val_fn0.5: 587247.0000 - val_precision0.5: 0.6162 - val_recall0.5: 0.8143 - val_tp0.7: 2325512.0000 - val_fp0.7: 840619.0000 - v
al_tn0.7: 15657327.0000 - val_fn0.7: 837342.0000 - val_precision0.7: 0.7345 - val_recall0.7: 0.7353 - val_tp0.9: 1883413.0000 - val_fp0.9: 305362.0000 - val_tn0.9: 16192584.0000 - val_fn0.
9: 1279441.0000 - val_precision0.9: 0.8605 - val_recall0.9: 0.5955 - val_accuracy: 0.8885 - val_auc: 0.9234 - val_f1: 0.2772
Epoch 4/20
300/300 [==============================] - 109s 364ms/step - loss: 0.7296 - tp0.1: 9976082.0000 - fp0.1: 10739149.0000 - tn0.1: 55934272.0000 - fn0.1: 1993685.0000 - precision0.1: 0.4816 -
 recall0.1: 0.8334 - tp0.3: 8926198.0000 - fp0.3: 4393527.0000 - tn0.3: 62279884.0000 - fn0.3: 3043569.0000 - precision0.3: 0.6701 - recall0.3: 0.7457 - tp0.5: 7905026.0000 - fp0.5: 217465
0.0000 - tn0.5: 64498764.0000 - fn0.5: 4064741.0000 - precision0.5: 0.7843 - recall0.5: 0.6604 - tp0.7: 6608527.0000 - fp0.7: 943676.0000 - tn0.7: 65729772.0000 - fn0.7: 5361240.0000 - pre
cision0.7: 0.8750 - recall0.7: 0.5521 - tp0.9: 4494816.0000 - fp0.9: 193397.0000 - tn0.9: 66480032.0000 - fn0.9: 7474951.0000 - precision0.9: 0.9587 - recall0.9: 0.3755 - accuracy: 0.9207
- auc: 0.8902 - f1: 0.2642 - val_loss: 0.9724 - val_tp0.1: 2946822.0000 - val_fp0.1: 7011140.0000 - val_tn0.1: 9486806.0000 - val_fn0.1: 216032.0000 - val_precision0.1: 0.2959 - val_recall
0.1: 0.9317 - val_tp0.3: 2828230.0000 - val_fp0.3: 4270134.0000 - val_tn0.3: 12227812.0000 - val_fn0.3: 334624.0000 - val_precision0.3: 0.3984 - val_recall0.3: 0.8942 - val_tp0.5: 2748329.
0000 - val_fp0.5: 3138657.0000 - val_tn0.5: 13359289.0000 - val_fn0.5: 414525.0000 - val_precision0.5: 0.4668 - val_recall0.5: 0.8689 - val_tp0.7: 2589983.0000 - val_fp0.7: 1878374.0000 -
val_tn0.7: 14619572.0000 - val_fn0.7: 572871.0000 - val_precision0.7: 0.5796 - val_recall0.7: 0.8189 - val_tp0.9: 2101517.0000 - val_fp0.9: 517482.0000 - val_tn0.9: 15980464.0000 - val_fn0
.9: 1061337.0000 - val_precision0.9: 0.8024 - val_recall0.9: 0.6644 - val_accuracy: 0.8193 - val_auc: 0.9128 - val_f1: 0.2772
Epoch 5/20
300/300 [==============================] - 109s 363ms/step - loss: 0.6529 - tp0.1: 10122261.0000 - fp0.1: 9591155.0000 - tn0.1: 57082268.0000 - fn0.1: 1847506.0000 - precision0.1: 0.5135 -
 recall0.1: 0.8457 - tp0.3: 9144708.0000 - fp0.3: 3989412.0000 - tn0.3: 62684028.0000 - fn0.3: 2825059.0000 - precision0.3: 0.6963 - recall0.3: 0.7640 - tp0.5: 8259383.0000 - fp0.5: 210977
9.0000 - tn0.5: 64563668.0000 - fn0.5: 3710384.0000 - precision0.5: 0.7965 - recall0.5: 0.6900 - tp0.7: 7094469.0000 - fp0.7: 988158.0000 - tn0.7: 65685280.0000 - fn0.7: 4875298.0000 - pre
cision0.7: 0.8777 - recall0.7: 0.5927 - tp0.9: 4946279.0000 - fp0.9: 215942.0000 - tn0.9: 66457496.0000 - fn0.9: 7023488.0000 - precision0.9: 0.9582 - recall0.9: 0.4132 - accuracy: 0.9260
- auc: 0.8999 - f1: 0.2642 - val_loss: 0.9432 - val_tp0.1: 3012100.0000 - val_fp0.1: 7946515.0000 - val_tn0.1: 8551431.0000 - val_fn0.1: 150754.0000 - val_precision0.1: 0.2749 - val_recall
0.1: 0.9523 - val_tp0.3: 2894544.0000 - val_fp0.3: 4511247.0000 - val_tn0.3: 11986699.0000 - val_fn0.3: 268310.0000 - val_precision0.3: 0.3908 - val_recall0.3: 0.9152 - val_tp0.5: 2779750.
0000 - val_fp0.5: 2798125.0000 - val_tn0.5: 13699821.0000 - val_fn0.5: 383104.0000 - val_precision0.5: 0.4984 - val_recall0.5: 0.8789 - val_tp0.7: 2629141.0000 - val_fp0.7: 1703471.0000 -
val_tn0.7: 14794475.0000 - val_fn0.7: 533713.0000 - val_precision0.7: 0.6068 - val_recall0.7: 0.8313 - val_tp0.9: 2267860.0000 - val_fp0.9: 655578.0000 - val_tn0.9: 15842368.0000 - val_fn0
.9: 894994.0000 - val_precision0.9: 0.7758 - val_recall0.9: 0.7170 - val_accuracy: 0.8382 - val_auc: 0.9243 - val_f1: 0.2772
Epoch 6/20
300/300 [==============================] - 109s 363ms/step - loss: 0.6248 - tp0.1: 10165542.0000 - fp0.1: 8993953.0000 - tn0.1: 57679492.0000 - fn0.1: 1804225.0000 - precision0.1: 0.5306 -
 recall0.1: 0.8493 - tp0.3: 9220264.0000 - fp0.3: 3729094.0000 - tn0.3: 62944360.0000 - fn0.3: 2749503.0000 - precision0.3: 0.7120 - recall0.3: 0.7703 - tp0.5: 8427138.0000 - fp0.5: 202054
3.0000 - tn0.5: 64652888.0000 - fn0.5: 3542629.0000 - precision0.5: 0.8066 - recall0.5: 0.7040 - tp0.7: 7328827.0000 - fp0.7: 939211.0000 - tn0.7: 65734200.0000 - fn0.7: 4640940.0000 - pre
cision0.7: 0.8864 - recall0.7: 0.6123 - tp0.9: 5167941.0000 - fp0.9: 189519.0000 - tn0.9: 66483928.0000 - fn0.9: 6801826.0000 - precision0.9: 0.9646 - recall0.9: 0.4317 - accuracy: 0.9293
- auc: 0.9044 - f1: 0.2642 - val_loss: 0.5511 - val_tp0.1: 2844793.0000 - val_fp0.1: 2570394.0000 - val_tn0.1: 13927552.0000 - val_fn0.1: 318061.0000 - val_precision0.1: 0.5253 - val_recal
l0.1: 0.8994 - val_tp0.3: 2684206.0000 - val_fp0.3: 1367923.0000 - val_tn0.3: 15130023.0000 - val_fn0.3: 478648.0000 - val_precision0.3: 0.6624 - val_recall0.3: 0.8487 - val_tp0.5: 2475186
.0000 - val_fp0.5: 730745.0000 - val_tn0.5: 15767201.0000 - val_fn0.5: 687668.0000 - val_precision0.5: 0.7721 - val_recall0.5: 0.7826 - val_tp0.7: 2218398.0000 - val_fp0.7: 388178.0000 - v
al_tn0.7: 16109768.0000 - val_fn0.7: 944456.0000 - val_precision0.7: 0.8511 - val_recall0.7: 0.7014 - val_tp0.9: 1713349.0000 - val_fp0.9: 114270.0000 - val_tn0.9: 16383676.0000 - val_fn0.
9: 1449505.0000 - val_precision0.9: 0.9375 - val_recall0.9: 0.5417 - val_accuracy: 0.9279 - val_auc: 0.9285 - val_f1: 0.2772
Epoch 7/20
300/300 [==============================] - 109s 364ms/step - loss: 0.6064 - tp0.1: 10229897.0000 - fp0.1: 8897750.0000 - tn0.1: 57775700.0000 - fn0.1: 1739870.0000 - precision0.1: 0.5348 -
 recall0.1: 0.8546 - tp0.3: 9283055.0000 - fp0.3: 3514175.0000 - tn0.3: 63159248.0000 - fn0.3: 2686712.0000 - precision0.3: 0.7254 - recall0.3: 0.7755 - tp0.5: 8533348.0000 - fp0.5: 192231
9.0000 - tn0.5: 64751124.0000 - fn0.5: 3436419.0000 - precision0.5: 0.8161 - recall0.5: 0.7129 - tp0.7: 7491425.0000 - fp0.7: 917374.0000 - tn0.7: 65756056.0000 - fn0.7: 4478342.0000 - pre
cision0.7: 0.8909 - recall0.7: 0.6259 - tp0.9: 5439611.0000 - fp0.9: 210848.0000 - tn0.9: 66462588.0000 - fn0.9: 6530156.0000 - precision0.9: 0.9627 - recall0.9: 0.4544 - accuracy: 0.9319
- auc: 0.9082 - f1: 0.2642 - val_loss: 0.6010 - val_tp0.1: 2934541.0000 - val_fp0.1: 3995941.0000 - val_tn0.1: 12502005.0000 - val_fn0.1: 228313.0000 - val_precision0.1: 0.4234 - val_recal
l0.1: 0.9278 - val_tp0.3: 2764037.0000 - val_fp0.3: 1908942.0000 - val_tn0.3: 14589004.0000 - val_fn0.3: 398817.0000 - val_precision0.3: 0.5915 - val_recall0.3: 0.8739 - val_tp0.5: 2611207
.0000 - val_fp0.5: 1127029.0000 - val_tn0.5: 15370917.0000 - val_fn0.5: 551647.0000 - val_precision0.5: 0.6985 - val_recall0.5: 0.8256 - val_tp0.7: 2388708.0000 - val_fp0.7: 602484.0000 -
val_tn0.7: 15895462.0000 - val_fn0.7: 774146.0000 - val_precision0.7: 0.7986 - val_recall0.7: 0.7552 - val_tp0.9: 1910700.0000 - val_fp0.9: 183899.0000 - val_tn0.9: 16314047.0000 - val_fn0
.9: 1252154.0000 - val_precision0.9: 0.9122 - val_recall0.9: 0.6041 - val_accuracy: 0.9146 - val_auc: 0.9353 - val_f1: 0.2772
Epoch 8/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5966 - tp0.1: 10258117.0000 - fp0.1: 8820090.0000 - tn0.1: 57853336.0000 - fn0.1: 1711650.0000 - precision0.1: 0.5377 -
 recall0.1: 0.8570 - tp0.3: 9332270.0000 - fp0.3: 3494563.0000 - tn0.3: 63178884.0000 - fn0.3: 2637497.0000 - precision0.3: 0.7276 - recall0.3: 0.7797 - tp0.5: 8607177.0000 - fp0.5: 193241
9.0000 - tn0.5: 64741044.0000 - fn0.5: 3362590.0000 - precision0.5: 0.8167 - recall0.5: 0.7191 - tp0.7: 7560035.0000 - fp0.7: 907227.0000 - tn0.7: 65766188.0000 - fn0.7: 4409732.0000 - pre
cision0.7: 0.8929 - recall0.7: 0.6316 - tp0.9: 5489005.0000 - fp0.9: 193323.0000 - tn0.9: 66480128.0000 - fn0.9: 6480762.0000 - precision0.9: 0.9660 - recall0.9: 0.4586 - accuracy: 0.9327
- auc: 0.9098 - f1: 0.2642 - val_loss: 0.5360 - val_tp0.1: 2840705.0000 - val_fp0.1: 2432393.0000 - val_tn0.1: 14065553.0000 - val_fn0.1: 322149.0000 - val_precision0.1: 0.5387 - val_recal
l0.1: 0.8981 - val_tp0.3: 2541562.0000 - val_fp0.3: 814687.0000 - val_tn0.3: 15683259.0000 - val_fn0.3: 621292.0000 - val_precision0.3: 0.7573 - val_recall0.3: 0.8036 - val_tp0.5: 2332487.
0000 - val_fp0.5: 450050.0000 - val_tn0.5: 16047896.0000 - val_fn0.5: 830367.0000 - val_precision0.5: 0.8383 - val_recall0.5: 0.7375 - val_tp0.7: 2027801.0000 - val_fp0.7: 205102.0000 - va
l_tn0.7: 16292844.0000 - val_fn0.7: 1135053.0000 - val_precision0.7: 0.9081 - val_recall0.7: 0.6411 - val_tp0.9: 1478341.0000 - val_fp0.9: 39136.0000 - val_tn0.9: 16458810.0000 - val_fn0.9
: 1684513.0000 - val_precision0.9: 0.9742 - val_recall0.9: 0.4674 - val_accuracy: 0.9349 - val_auc: 0.9309 - val_f1: 0.2772
Epoch 9/20
300/300 [==============================] - 109s 363ms/step - loss: 0.5815 - tp0.1: 10317877.0000 - fp0.1: 8787663.0000 - tn0.1: 57885784.0000 - fn0.1: 1651890.0000 - precision0.1: 0.5400 -
 recall0.1: 0.8620 - tp0.3: 9391866.0000 - fp0.3: 3385982.0000 - tn0.3: 63287432.0000 - fn0.3: 2577901.0000 - precision0.3: 0.7350 - recall0.3: 0.7846 - tp0.5: 8683886.0000 - fp0.5: 185537
9.0000 - tn0.5: 64818064.0000 - fn0.5: 3285881.0000 - precision0.5: 0.8240 - recall0.5: 0.7255 - tp0.7: 7664640.0000 - fp0.7: 872040.0000 - tn0.7: 65801404.0000 - fn0.7: 4305127.0000 - pre
cision0.7: 0.8978 - recall0.7: 0.6403 - tp0.9: 5645583.0000 - fp0.9: 192263.0000 - tn0.9: 66481188.0000 - fn0.9: 6324184.0000 - precision0.9: 0.9671 - recall0.9: 0.4717 - accuracy: 0.9346
- auc: 0.9130 - f1: 0.2642 - val_loss: 0.5508 - val_tp0.1: 2890310.0000 - val_fp0.1: 3133203.0000 - val_tn0.1: 13364743.0000 - val_fn0.1: 272544.0000 - val_precision0.1: 0.4798 - val_recal
l0.1: 0.9138 - val_tp0.3: 2582821.0000 - val_fp0.3: 941605.0000 - val_tn0.3: 15556341.0000 - val_fn0.3: 580033.0000 - val_precision0.3: 0.7328 - val_recall0.3: 0.8166 - val_tp0.5: 2342586.
0000 - val_fp0.5: 468856.0000 - val_tn0.5: 16029090.0000 - val_fn0.5: 820268.0000 - val_precision0.5: 0.8332 - val_recall0.5: 0.7407 - val_tp0.7: 2000631.0000 - val_fp0.7: 196312.0000 - va
l_tn0.7: 16301634.0000 - val_fn0.7: 1162223.0000 - val_precision0.7: 0.9106 - val_recall0.7: 0.6325 - val_tp0.9: 1393863.0000 - val_fp0.9: 35445.0000 - val_tn0.9: 16462501.0000 - val_fn0.9
: 1768991.0000 - val_precision0.9: 0.9752 - val_recall0.9: 0.4407 - val_accuracy: 0.9344 - val_auc: 0.9339 - val_f1: 0.2772
Epoch 10/20
300/300 [==============================] - 109s 363ms/step - loss: 0.5742 - tp0.1: 10342521.0000 - fp0.1: 8755470.0000 - tn0.1: 57917940.0000 - fn0.1: 1627246.0000 - precision0.1: 0.5416 -
 recall0.1: 0.8641 - tp0.3: 9412910.0000 - fp0.3: 3294713.0000 - tn0.3: 63378728.0000 - fn0.3: 2556857.0000 - precision0.3: 0.7407 - recall0.3: 0.7864 - tp0.5: 8709465.0000 - fp0.5: 179218
3.0000 - tn0.5: 64881220.0000 - fn0.5: 3260302.0000 - precision0.5: 0.8293 - recall0.5: 0.7276 - tp0.7: 7716001.0000 - fp0.7: 846147.0000 - tn0.7: 65827296.0000 - fn0.7: 4253766.0000 - pre
cision0.7: 0.9012 - recall0.7: 0.6446 - tp0.9: 5786134.0000 - fp0.9: 193748.0000 - tn0.9: 66479688.0000 - fn0.9: 6183633.0000 - precision0.9: 0.9676 - recall0.9: 0.4834 - accuracy: 0.9358
- auc: 0.9142 - f1: 0.2642 - val_loss: 0.5714 - val_tp0.1: 2695380.0000 - val_fp0.1: 1362499.0000 - val_tn0.1: 15135447.0000 - val_fn0.1: 467474.0000 - val_precision0.1: 0.6642 - val_recal
l0.1: 0.8522 - val_tp0.3: 2331121.0000 - val_fp0.3: 439292.0000 - val_tn0.3: 16058654.0000 - val_fn0.3: 831733.0000 - val_precision0.3: 0.8414 - val_recall0.3: 0.7370 - val_tp0.5: 2008752.
0000 - val_fp0.5: 188756.0000 - val_tn0.5: 16309190.0000 - val_fn0.5: 1154102.0000 - val_precision0.5: 0.9141 - val_recall0.5: 0.6351 - val_tp0.7: 1582146.0000 - val_fp0.7: 59586.0000 - va
l_tn0.7: 16438360.0000 - val_fn0.7: 1580708.0000 - val_precision0.7: 0.9637 - val_recall0.7: 0.5002 - val_tp0.9: 896176.0000 - val_fp0.9: 4415.0000 - val_tn0.9: 16493531.0000 - val_fn0.9:
2266678.0000 - val_precision0.9: 0.9951 - val_recall0.9: 0.2833 - val_accuracy: 0.9317 - val_auc: 0.9152 - val_f1: 0.2772
Epoch 11/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5695 - tp0.1: 10353763.0000 - fp0.1: 8713907.0000 - tn0.1: 57959528.0000 - fn0.1: 1616004.0000 - precision0.1: 0.5430 -
 recall0.1: 0.8650 - tp0.3: 9444896.0000 - fp0.3: 3321969.0000 - tn0.3: 63351480.0000 - fn0.3: 2524871.0000 - precision0.3: 0.7398 - recall0.3: 0.7891 - tp0.5: 8747813.0000 - fp0.5: 180398
4.0000 - tn0.5: 64869452.0000 - fn0.5: 3221954.0000 - precision0.5: 0.8290 - recall0.5: 0.7308 - tp0.7: 7782574.0000 - fp0.7: 853915.0000 - tn0.7: 65819536.0000 - fn0.7: 4187193.0000 - pre
cision0.7: 0.9011 - recall0.7: 0.6502 - tp0.9: 5870371.0000 - fp0.9: 194465.0000 - tn0.9: 66479000.0000 - fn0.9: 6099396.0000 - precision0.9: 0.9679 - recall0.9: 0.4904 - accuracy: 0.9361
- auc: 0.9151 - f1: 0.2642 - val_loss: 0.5577 - val_tp0.1: 2892745.0000 - val_fp0.1: 3143327.0000 - val_tn0.1: 13354619.0000 - val_fn0.1: 270109.0000 - val_precision0.1: 0.4792 - val_recal
l0.1: 0.9146 - val_tp0.3: 2706166.0000 - val_fp0.3: 1385967.0000 - val_tn0.3: 15111979.0000 - val_fn0.3: 456688.0000 - val_precision0.3: 0.6613 - val_recall0.3: 0.8556 - val_tp0.5: 2567444
.0000 - val_fp0.5: 857066.0000 - val_tn0.5: 15640880.0000 - val_fn0.5: 595410.0000 - val_precision0.5: 0.7497 - val_recall0.5: 0.8117 - val_tp0.7: 2361482.0000 - val_fp0.7: 478495.0000 - v
al_tn0.7: 16019451.0000 - val_fn0.7: 801372.0000 - val_precision0.7: 0.8315 - val_recall0.7: 0.7466 - val_tp0.9: 1980797.0000 - val_fp0.9: 176419.0000 - val_tn0.9: 16321527.0000 - val_fn0.
9: 1182057.0000 - val_precision0.9: 0.9182 - val_recall0.9: 0.6263 - val_accuracy: 0.9261 - val_auc: 0.9342 - val_f1: 0.2772
Epoch 12/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5631 - tp0.1: 10359789.0000 - fp0.1: 8488876.0000 - tn0.1: 58184560.0000 - fn0.1: 1609978.0000 - precision0.1: 0.5496 -
 recall0.1: 0.8655 - tp0.3: 9469292.0000 - fp0.3: 3249710.0000 - tn0.3: 63423736.0000 - fn0.3: 2500475.0000 - precision0.3: 0.7445 - recall0.3: 0.7911 - tp0.5: 8773908.0000 - fp0.5: 173654
4.0000 - tn0.5: 64936880.0000 - fn0.5: 3195859.0000 - precision0.5: 0.8348 - recall0.5: 0.7330 - tp0.7: 7799121.0000 - fp0.7: 807785.0000 - tn0.7: 65865644.0000 - fn0.7: 4170646.0000 - pre
cision0.7: 0.9061 - recall0.7: 0.6516 - tp0.9: 5889535.0000 - fp0.9: 186306.0000 - tn0.9: 66487124.0000 - fn0.9: 6080232.0000 - precision0.9: 0.9693 - recall0.9: 0.4920 - accuracy: 0.9373
- auc: 0.9159 - f1: 0.2642 - val_loss: 0.5163 - val_tp0.1: 2823752.0000 - val_fp0.1: 2094434.0000 - val_tn0.1: 14403512.0000 - val_fn0.1: 339102.0000 - val_precision0.1: 0.5741 - val_recal
l0.1: 0.8928 - val_tp0.3: 2583025.0000 - val_fp0.3: 825425.0000 - val_tn0.3: 15672521.0000 - val_fn0.3: 579829.0000 - val_precision0.3: 0.7578 - val_recall0.3: 0.8167 - val_tp0.5: 2379325.
0000 - val_fp0.5: 447908.0000 - val_tn0.5: 16050038.0000 - val_fn0.5: 783529.0000 - val_precision0.5: 0.8416 - val_recall0.5: 0.7523 - val_tp0.7: 2087728.0000 - val_fp0.7: 207419.0000 - va
l_tn0.7: 16290527.0000 - val_fn0.7: 1075126.0000 - val_precision0.7: 0.9096 - val_recall0.7: 0.6601 - val_tp0.9: 1527766.0000 - val_fp0.9: 40879.0000 - val_tn0.9: 16457067.0000 - val_fn0.9
: 1635088.0000 - val_precision0.9: 0.9739 - val_recall0.9: 0.4830 - val_accuracy: 0.9374 - val_auc: 0.9303 - val_f1: 0.2772
Epoch 13/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5583 - tp0.1: 10369243.0000 - fp0.1: 8417163.0000 - tn0.1: 58256252.0000 - fn0.1: 1600524.0000 - precision0.1: 0.5520 -
 recall0.1: 0.8663 - tp0.3: 9495979.0000 - fp0.3: 3288261.0000 - tn0.3: 63385168.0000 - fn0.3: 2473788.0000 - precision0.3: 0.7428 - recall0.3: 0.7933 - tp0.5: 8802292.0000 - fp0.5: 176251
9.0000 - tn0.5: 64910900.0000 - fn0.5: 3167475.0000 - precision0.5: 0.8332 - recall0.5: 0.7354 - tp0.7: 7853085.0000 - fp0.7: 831023.0000 - tn0.7: 65842424.0000 - fn0.7: 4116682.0000 - pre
cision0.7: 0.9043 - recall0.7: 0.6561 - tp0.9: 5967523.0000 - fp0.9: 191510.0000 - tn0.9: 66481912.0000 - fn0.9: 6002244.0000 - precision0.9: 0.9689 - recall0.9: 0.4985 - accuracy: 0.9373
- auc: 0.9166 - f1: 0.2642 - val_loss: 0.5155 - val_tp0.1: 2810372.0000 - val_fp0.1: 1944105.0000 - val_tn0.1: 14553841.0000 - val_fn0.1: 352482.0000 - val_precision0.1: 0.5911 - val_recal
l0.1: 0.8886 - val_tp0.3: 2550568.0000 - val_fp0.3: 737116.0000 - val_tn0.3: 15760830.0000 - val_fn0.3: 612286.0000 - val_precision0.3: 0.7758 - val_recall0.3: 0.8064 - val_tp0.5: 2338331.
0000 - val_fp0.5: 398618.0000 - val_tn0.5: 16099328.0000 - val_fn0.5: 824523.0000 - val_precision0.5: 0.8544 - val_recall0.5: 0.7393 - val_tp0.7: 2056023.0000 - val_fp0.7: 191281.0000 - va
l_tn0.7: 16306665.0000 - val_fn0.7: 1106831.0000 - val_precision0.7: 0.9149 - val_recall0.7: 0.6501 - val_tp0.9: 1536052.0000 - val_fp0.9: 42617.0000 - val_tn0.9: 16455329.0000 - val_fn0.9
: 1626802.0000 - val_precision0.9: 0.9730 - val_recall0.9: 0.4857 - val_accuracy: 0.9378 - val_auc: 0.9292 - val_f1: 0.2772
Epoch 14/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5545 - tp0.1: 10404673.0000 - fp0.1: 8594836.0000 - tn0.1: 58078576.0000 - fn0.1: 1565094.0000 - precision0.1: 0.5476 -
 recall0.1: 0.8692 - tp0.3: 9502698.0000 - fp0.3: 3265228.0000 - tn0.3: 63408216.0000 - fn0.3: 2467069.0000 - precision0.3: 0.7443 - recall0.3: 0.7939 - tp0.5: 8806002.0000 - fp0.5: 173452
7.0000 - tn0.5: 64938912.0000 - fn0.5: 3163765.0000 - precision0.5: 0.8354 - recall0.5: 0.7357 - tp0.7: 7871330.0000 - fp0.7: 811976.0000 - tn0.7: 65861468.0000 - fn0.7: 4098437.0000 - pre
cision0.7: 0.9065 - recall0.7: 0.6576 - tp0.9: 5995175.0000 - fp0.9: 182094.0000 - tn0.9: 66491348.0000 - fn0.9: 5974592.0000 - precision0.9: 0.9705 - recall0.9: 0.5009 - accuracy: 0.9377
- auc: 0.9180 - f1: 0.2642 - val_loss: 0.5147 - val_tp0.1: 2829492.0000 - val_fp0.1: 2130176.0000 - val_tn0.1: 14367770.0000 - val_fn0.1: 333362.0000 - val_precision0.1: 0.5705 - val_recal
l0.1: 0.8946 - val_tp0.3: 2583127.0000 - val_fp0.3: 813224.0000 - val_tn0.3: 15684722.0000 - val_fn0.3: 579727.0000 - val_precision0.3: 0.7606 - val_recall0.3: 0.8167 - val_tp0.5: 2383747.
0000 - val_fp0.5: 446660.0000 - val_tn0.5: 16051286.0000 - val_fn0.5: 779107.0000 - val_precision0.5: 0.8422 - val_recall0.5: 0.7537 - val_tp0.7: 2116906.0000 - val_fp0.7: 219992.0000 - va
l_tn0.7: 16277954.0000 - val_fn0.7: 1045948.0000 - val_precision0.7: 0.9059 - val_recall0.7: 0.6693 - val_tp0.9: 1607900.0000 - val_fp0.9: 51317.0000 - val_tn0.9: 16446629.0000 - val_fn0.9
: 1554954.0000 - val_precision0.9: 0.9691 - val_recall0.9: 0.5084 - val_accuracy: 0.9377 - val_auc: 0.9312 - val_f1: 0.2772
Epoch 15/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5536 - tp0.1: 10410305.0000 - fp0.1: 8632096.0000 - tn0.1: 58041320.0000 - fn0.1: 1559462.0000 - precision0.1: 0.5467 -
 recall0.1: 0.8697 - tp0.3: 9510984.0000 - fp0.3: 3242903.0000 - tn0.3: 63430520.0000 - fn0.3: 2458783.0000 - precision0.3: 0.7457 - recall0.3: 0.7946 - tp0.5: 8812511.0000 - fp0.5: 170138
1.0000 - tn0.5: 64972056.0000 - fn0.5: 3157256.0000 - precision0.5: 0.8382 - recall0.5: 0.7362 - tp0.7: 7877931.0000 - fp0.7: 793039.0000 - tn0.7: 65880400.0000 - fn0.7: 4091836.0000 - pre
cision0.7: 0.9085 - recall0.7: 0.6582 - tp0.9: 5978990.0000 - fp0.9: 174678.0000 - tn0.9: 66498776.0000 - fn0.9: 5990777.0000 - precision0.9: 0.9716 - recall0.9: 0.4995 - accuracy: 0.9382
- auc: 0.9183 - f1: 0.2642 - val_loss: 0.5193 - val_tp0.1: 2813989.0000 - val_fp0.1: 2032257.0000 - val_tn0.1: 14465689.0000 - val_fn0.1: 348865.0000 - val_precision0.1: 0.5807 - val_recal
l0.1: 0.8897 - val_tp0.3: 2577154.0000 - val_fp0.3: 819794.0000 - val_tn0.3: 15678152.0000 - val_fn0.3: 585700.0000 - val_precision0.3: 0.7587 - val_recall0.3: 0.8148 - val_tp0.5: 2385705.
0000 - val_fp0.5: 459176.0000 - val_tn0.5: 16038770.0000 - val_fn0.5: 777149.0000 - val_precision0.5: 0.8386 - val_recall0.5: 0.7543 - val_tp0.7: 2130654.0000 - val_fp0.7: 230334.0000 - va
l_tn0.7: 16267612.0000 - val_fn0.7: 1032200.0000 - val_precision0.7: 0.9024 - val_recall0.7: 0.6736 - val_tp0.9: 1640662.0000 - val_fp0.9: 55692.0000 - val_tn0.9: 16442254.0000 - val_fn0.9
: 1522192.0000 - val_precision0.9: 0.9672 - val_recall0.9: 0.5187 - val_accuracy: 0.9371 - val_auc: 0.9291 - val_f1: 0.2772
Epoch 16/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5574 - tp0.1: 10366848.0000 - fp0.1: 8351245.0000 - tn0.1: 58322184.0000 - fn0.1: 1602919.0000 - precision0.1: 0.5538 -
 recall0.1: 0.8661 - tp0.3: 9482187.0000 - fp0.3: 3195743.0000 - tn0.3: 63477668.0000 - fn0.3: 2487580.0000 - precision0.3: 0.7479 - recall0.3: 0.7922 - tp0.5: 8790251.0000 - fp0.5: 169760
0.0000 - tn0.5: 64975828.0000 - fn0.5: 3179516.0000 - precision0.5: 0.8381 - recall0.5: 0.7344 - tp0.7: 7865829.0000 - fp0.7: 803359.0000 - tn0.7: 65870080.0000 - fn0.7: 4103938.0000 - pre
cision0.7: 0.9073 - recall0.7: 0.6571 - tp0.9: 6012200.0000 - fp0.9: 181018.0000 - tn0.9: 66492408.0000 - fn0.9: 5957567.0000 - precision0.9: 0.9708 - recall0.9: 0.5023 - accuracy: 0.9380
- auc: 0.9168 - f1: 0.2642 - val_loss: 0.5190 - val_tp0.1: 2815670.0000 - val_fp0.1: 2047710.0000 - val_tn0.1: 14450236.0000 - val_fn0.1: 347184.0000 - val_precision0.1: 0.5790 - val_recal
l0.1: 0.8902 - val_tp0.3: 2576965.0000 - val_fp0.3: 816090.0000 - val_tn0.3: 15681856.0000 - val_fn0.3: 585889.0000 - val_precision0.3: 0.7595 - val_recall0.3: 0.8148 - val_tp0.5: 2382191.
0000 - val_fp0.5: 452866.0000 - val_tn0.5: 16045080.0000 - val_fn0.5: 780663.0000 - val_precision0.5: 0.8403 - val_recall0.5: 0.7532 - val_tp0.7: 2122711.0000 - val_fp0.7: 224345.0000 - va
l_tn0.7: 16273601.0000 - val_fn0.7: 1040143.0000 - val_precision0.7: 0.9044 - val_recall0.7: 0.6711 - val_tp0.9: 1623419.0000 - val_fp0.9: 52719.0000 - val_tn0.9: 16445227.0000 - val_fn0.9
: 1539435.0000 - val_precision0.9: 0.9685 - val_recall0.9: 0.5133 - val_accuracy: 0.9373 - val_auc: 0.9293 - val_f1: 0.2772
Epoch 17/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5518 - tp0.1: 10385606.0000 - fp0.1: 8322065.0000 - tn0.1: 58351360.0000 - fn0.1: 1584161.0000 - precision0.1: 0.5552 -
 recall0.1: 0.8677 - tp0.3: 9501942.0000 - fp0.3: 3183126.0000 - tn0.3: 63490304.0000 - fn0.3: 2467825.0000 - precision0.3: 0.7491 - recall0.3: 0.7938 - tp0.5: 8803116.0000 - fp0.5: 167708
6.0000 - tn0.5: 64996332.0000 - fn0.5: 3166651.0000 - precision0.5: 0.8400 - recall0.5: 0.7354 - tp0.7: 7870926.0000 - fp0.7: 783130.0000 - tn0.7: 65890296.0000 - fn0.7: 4098841.0000 - pre
cision0.7: 0.9095 - recall0.7: 0.6576 - tp0.9: 5981676.0000 - fp0.9: 175979.0000 - tn0.9: 66497456.0000 - fn0.9: 5988091.0000 - precision0.9: 0.9714 - recall0.9: 0.4997 - accuracy: 0.9384
- auc: 0.9178 - f1: 0.2642 - val_loss: 0.5176 - val_tp0.1: 2814391.0000 - val_fp0.1: 2019842.0000 - val_tn0.1: 14478104.0000 - val_fn0.1: 348463.0000 - val_precision0.1: 0.5822 - val_recal
l0.1: 0.8898 - val_tp0.3: 2575426.0000 - val_fp0.3: 804457.0000 - val_tn0.3: 15693489.0000 - val_fn0.3: 587428.0000 - val_precision0.3: 0.7620 - val_recall0.3: 0.8143 - val_tp0.5: 2377184.
0000 - val_fp0.5: 443147.0000 - val_tn0.5: 16054799.0000 - val_fn0.5: 785670.0000 - val_precision0.5: 0.8429 - val_recall0.5: 0.7516 - val_tp0.7: 2111101.0000 - val_fp0.7: 215747.0000 - va
l_tn0.7: 16282199.0000 - val_fn0.7: 1051753.0000 - val_precision0.7: 0.9073 - val_recall0.7: 0.6675 - val_tp0.9: 1598659.0000 - val_fp0.9: 48117.0000 - val_tn0.9: 16449829.0000 - val_fn0.9
: 1564195.0000 - val_precision0.9: 0.9708 - val_recall0.9: 0.5054 - val_accuracy: 0.9375 - val_auc: 0.9293 - val_f1: 0.2772
Epoch 18/20
300/300 [==============================] - 109s 365ms/step - loss: 0.5527 - tp0.1: 10394599.0000 - fp0.1: 8422625.0000 - tn0.1: 58250828.0000 - fn0.1: 1575168.0000 - precision0.1: 0.5524 -
 recall0.1: 0.8684 - tp0.3: 9515719.0000 - fp0.3: 3245634.0000 - tn0.3: 63427764.0000 - fn0.3: 2454048.0000 - precision0.3: 0.7457 - recall0.3: 0.7950 - tp0.5: 8829115.0000 - fp0.5: 172580
7.0000 - tn0.5: 64947624.0000 - fn0.5: 3140652.0000 - precision0.5: 0.8365 - recall0.5: 0.7376 - tp0.7: 7902699.0000 - fp0.7: 815002.0000 - tn0.7: 65858428.0000 - fn0.7: 4067068.0000 - pre
cision0.7: 0.9065 - recall0.7: 0.6602 - tp0.9: 5992723.0000 - fp0.9: 181617.0000 - tn0.9: 66491804.0000 - fn0.9: 5977044.0000 - precision0.9: 0.9706 - recall0.9: 0.5007 - accuracy: 0.9381
- auc: 0.9180 - f1: 0.2642 - val_loss: 0.5170 - val_tp0.1: 2818280.0000 - val_fp0.1: 2051127.0000 - val_tn0.1: 14446819.0000 - val_fn0.1: 344574.0000 - val_precision0.1: 0.5788 - val_recal
l0.1: 0.8911 - val_tp0.3: 2582361.0000 - val_fp0.3: 822058.0000 - val_tn0.3: 15675888.0000 - val_fn0.3: 580493.0000 - val_precision0.3: 0.7585 - val_recall0.3: 0.8165 - val_tp0.5: 2386572.
0000 - val_fp0.5: 453718.0000 - val_tn0.5: 16044228.0000 - val_fn0.5: 776282.0000 - val_precision0.5: 0.8403 - val_recall0.5: 0.7546 - val_tp0.7: 2123293.0000 - val_fp0.7: 221964.0000 - va
l_tn0.7: 16275982.0000 - val_fn0.7: 1039561.0000 - val_precision0.7: 0.9054 - val_recall0.7: 0.6713 - val_tp0.9: 1613816.0000 - val_fp0.9: 50399.0000 - val_tn0.9: 16447547.0000 - val_fn0.9
: 1549038.0000 - val_precision0.9: 0.9697 - val_recall0.9: 0.5102 - val_accuracy: 0.9374 - val_auc: 0.9297 - val_f1: 0.2772
Epoch 19/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5502 - tp0.1: 10400846.0000 - fp0.1: 8405890.0000 - tn0.1: 58267520.0000 - fn0.1: 1568921.0000 - precision0.1: 0.5530 -
 recall0.1: 0.8689 - tp0.3: 9521394.0000 - fp0.3: 3231081.0000 - tn0.3: 63442368.0000 - fn0.3: 2448373.0000 - precision0.3: 0.7466 - recall0.3: 0.7955 - tp0.5: 8830551.0000 - fp0.5: 170937
1.0000 - tn0.5: 64964080.0000 - fn0.5: 3139216.0000 - precision0.5: 0.8378 - recall0.5: 0.7377 - tp0.7: 7900201.0000 - fp0.7: 803691.0000 - tn0.7: 65869728.0000 - fn0.7: 4069566.0000 - pre
cision0.7: 0.9077 - recall0.7: 0.6600 - tp0.9: 5995260.0000 - fp0.9: 181599.0000 - tn0.9: 66491848.0000 - fn0.9: 5974507.0000 - precision0.9: 0.9706 - recall0.9: 0.5009 - accuracy: 0.9383
- auc: 0.9184 - f1: 0.2642 - val_loss: 0.5175 - val_tp0.1: 2819063.0000 - val_fp0.1: 2064294.0000 - val_tn0.1: 14433652.0000 - val_fn0.1: 343791.0000 - val_precision0.1: 0.5773 - val_recal
l0.1: 0.8913 - val_tp0.3: 2584585.0000 - val_fp0.3: 828069.0000 - val_tn0.3: 15669877.0000 - val_fn0.3: 578269.0000 - val_precision0.3: 0.7574 - val_recall0.3: 0.8172 - val_tp0.5: 2390134.
0000 - val_fp0.5: 458519.0000 - val_tn0.5: 16039427.0000 - val_fn0.5: 772720.0000 - val_precision0.5: 0.8390 - val_recall0.5: 0.7557 - val_tp0.7: 2128769.0000 - val_fp0.7: 225470.0000 - va
l_tn0.7: 16272476.0000 - val_fn0.7: 1034085.0000 - val_precision0.7: 0.9042 - val_recall0.7: 0.6731 - val_tp0.9: 1617794.0000 - val_fp0.9: 50968.0000 - val_tn0.9: 16446978.0000 - val_fn0.9
: 1545060.0000 - val_precision0.9: 0.9695 - val_recall0.9: 0.5115 - val_accuracy: 0.9374 - val_auc: 0.9298 - val_f1: 0.2772
Epoch 20/20
300/300 [==============================] - 109s 364ms/step - loss: 0.5540 - tp0.1: 10388304.0000 - fp0.1: 8427637.0000 - tn0.1: 58245784.0000 - fn0.1: 1581463.0000 - precision0.1: 0.5521 -
 recall0.1: 0.8679 - tp0.3: 9510729.0000 - fp0.3: 3244455.0000 - tn0.3: 63429016.0000 - fn0.3: 2459038.0000 - precision0.3: 0.7456 - recall0.3: 0.7946 - tp0.5: 8826300.0000 - fp0.5: 173006
6.0000 - tn0.5: 64943372.0000 - fn0.5: 3143467.0000 - precision0.5: 0.8361 - recall0.5: 0.7374 - tp0.7: 7907765.0000 - fp0.7: 815687.0000 - tn0.7: 65857736.0000 - fn0.7: 4062002.0000 - pre
cision0.7: 0.9065 - recall0.7: 0.6606 - tp0.9: 6007828.0000 - fp0.9: 181663.0000 - tn0.9: 66491748.0000 - fn0.9: 5961939.0000 - precision0.9: 0.9706 - recall0.9: 0.5019 - accuracy: 0.9380
- auc: 0.9177 - f1: 0.2642 - val_loss: 0.5180 - val_tp0.1: 2817068.0000 - val_fp0.1: 2045853.0000 - val_tn0.1: 14452093.0000 - val_fn0.1: 345786.0000 - val_precision0.1: 0.5793 - val_recal
l0.1: 0.8907 - val_tp0.3: 2578834.0000 - val_fp0.3: 815796.0000 - val_tn0.3: 15682150.0000 - val_fn0.3: 584020.0000 - val_precision0.3: 0.7597 - val_recall0.3: 0.8154 - val_tp0.5: 2382089.
0000 - val_fp0.5: 450657.0000 - val_tn0.5: 16047289.0000 - val_fn0.5: 780765.0000 - val_precision0.5: 0.8409 - val_recall0.5: 0.7531 - val_tp0.7: 2118583.0000 - val_fp0.7: 220753.0000 - va
l_tn0.7: 16277193.0000 - val_fn0.7: 1044271.0000 - val_precision0.7: 0.9056 - val_recall0.7: 0.6698 - val_tp0.9: 1609924.0000 - val_fp0.9: 50267.0000 - val_tn0.9: 16447679.0000 - val_fn0.9
: 1552930.0000 - val_precision0.9: 0.9697 - val_recall0.9: 0.5090 - val_accuracy: 0.9374 - val_auc: 0.9295 - val_f1: 0.2772
--- Running training session 59/140
{'hp_epochs': 20, 'hp_batch_size': 6, 'hp_scaler': 'maxabs', 'hp_n_levels': 3, 'hp_first_filters': 16, 'hp_pool_size': 2, 'hp_input_size': 4096, 'hp_lr_start': 0.06810286704342322, 'hp_lr_
power': 5.0}
--- repeat #: 1
input - shape:   (None, 4096, 1)
output - shape:  (None, 4096, 1)
Epoch 1/20
800/800 [==============================] - 40s 33ms/step - loss: 1.2638 - tp0.1: 2568942.0000 - fp0.1: 8167222.0000 - tn0.1: 8515746.0000 - fn0.1: 408890.0000 - precision0.1: 0.2393 - reca
ll0.1: 0.8627 - tp0.3: 1525563.0000 - fp0.3: 1857776.0000 - tn0.3: 14825192.0000 - fn0.3: 1452269.0000 - precision0.3: 0.4509 - recall0.3: 0.5123 - tp0.5: 952545.0000 - fp0.5: 543268.0000
- tn0.5: 16139700.0000 - fn0.5: 2025287.0000 - precision0.5: 0.6368 - recall0.5: 0.3199 - tp0.7: 600606.0000 - fp0.7: 171741.0000 - tn0.7: 16511227.0000 - fn0.7: 2377226.0000 - precision0.
7: 0.7776 - recall0.7: 0.2017 - tp0.9: 266111.0000 - fp0.9: 28495.0000 - tn0.9: 16654473.0000 - fn0.9: 2711721.0000 - precision0.9: 0.9033 - recall0.9: 0.0894 - accuracy: 0.8694 - auc: 0.7
995 - f1: 0.2631 - val_loss: 3.2184 - val_tp0.1: 785569.0000 - val_fp0.1: 3481300.0000 - val_tn0.1: 630043.0000 - val_fn0.1: 18288.0000 - val_precision0.1: 0.1841 - val_recall0.1: 0.9772 -
 val_tp0.3: 774801.0000 - val_fp0.3: 3330916.0000 - val_tn0.3: 780427.0000 - val_fn0.3: 29056.0000 - val_precision0.3: 0.1887 - val_recall0.3: 0.9639 - val_tp0.5: 766485.0000 - val_fp0.5:
3233476.0000 - val_tn0.5: 877867.0000 - val_fn0.5: 37372.0000 - val_precision0.5: 0.1916 - val_recall0.5: 0.9535 - val_tp0.7: 754282.0000 - val_fp0.7: 3093977.0000 - val_tn0.7: 1017366.000
0 - val_fn0.7: 49575.0000 - val_precision0.7: 0.1960 - val_recall0.7: 0.9383 - val_tp0.9: 704600.0000 - val_fp0.9: 2525773.0000 - val_tn0.9: 1585570.0000 - val_fn0.9: 99257.0000 - val_prec
ision0.9: 0.2181 - val_recall0.9: 0.8765 - val_accuracy: 0.3345 - val_auc: 0.7196 - val_f1: 0.2811
Epoch 2/20
800/800 [==============================] - 23s 28ms/step - loss: 1.0608 - tp0.1: 2627985.0000 - fp0.1: 7509687.0000 - tn0.1: 9173281.0000 - fn0.1: 349847.0000 - precision0.1: 0.2592 - reca
ll0.1: 0.8825 - tp0.3: 1657396.0000 - fp0.3: 1642042.0000 - tn0.3: 15040926.0000 - fn0.3: 1320436.0000 - precision0.3: 0.5023 - recall0.3: 0.5566 - tp0.5: 1186375.0000 - fp0.5: 597404.0000
 - tn0.5: 16085564.0000 - fn0.5: 1791457.0000 - precision0.5: 0.6651 - recall0.5: 0.3984 - tp0.7: 796587.0000 - fp0.7: 190824.0000 - tn0.7: 16492144.0000 - fn0.7: 2181245.0000 - precision0
.7: 0.8067 - recall0.7: 0.2675 - tp0.9: 382973.0000 - fp0.9: 30617.0000 - tn0.9: 16652351.0000 - fn0.9: 2594859.0000 - precision0.9: 0.9260 - recall0.9: 0.1286 - accuracy: 0.8785 - auc: 0.
8268 - f1: 0.2631 - val_loss: 2.8384 - val_tp0.1: 770043.0000 - val_fp0.1: 3359592.0000 - val_tn0.1: 751751.0000 - val_fn0.1: 33814.0000 - val_precision0.1: 0.1865 - val_recall0.1: 0.9579
- val_tp0.3: 728149.0000 - val_fp0.3: 3009354.0000 - val_tn0.3: 1101989.0000 - val_fn0.3: 75708.0000 - val_precision0.3: 0.1948 - val_recall0.3: 0.9058 - val_tp0.5: 698943.0000 - val_fp0.5
: 2825281.0000 - val_tn0.5: 1286062.0000 - val_fn0.5: 104914.0000 - val_precision0.5: 0.1983 - val_recall0.5: 0.8695 - val_tp0.7: 650069.0000 - val_fp0.7: 2558191.0000 - val_tn0.7: 1553152
.0000 - val_fn0.7: 153788.0000 - val_precision0.7: 0.2026 - val_recall0.7: 0.8087 - val_tp0.9: 523649.0000 - val_fp0.9: 1973892.0000 - val_tn0.9: 2137451.0000 - val_fn0.9: 280208.0000 - va
l_precision0.9: 0.2097 - val_recall0.9: 0.6514 - val_accuracy: 0.4039 - val_auc: 0.6320 - val_f1: 0.2811
Epoch 3/20
800/800 [==============================] - 22s 28ms/step - loss: 0.9210 - tp0.1: 2627224.0000 - fp0.1: 6591219.0000 - tn0.1: 10091749.0000 - fn0.1: 350608.0000 - precision0.1: 0.2850 - rec
all0.1: 0.8823 - tp0.3: 1806105.0000 - fp0.3: 1745398.0000 - tn0.3: 14937570.0000 - fn0.3: 1171727.0000 - precision0.3: 0.5085 - recall0.3: 0.6065 - tp0.5: 1334712.0000 - fp0.5: 661778.000
0 - tn0.5: 16021190.0000 - fn0.5: 1643120.0000 - precision0.5: 0.6685 - recall0.5: 0.4482 - tp0.7: 898005.0000 - fp0.7: 202721.0000 - tn0.7: 16480247.0000 - fn0.7: 2079827.0000 - precision
0.7: 0.8158 - recall0.7: 0.3016 - tp0.9: 425030.0000 - fp0.9: 26905.0000 - tn0.9: 16656063.0000 - fn0.9: 2552802.0000 - precision0.9: 0.9405 - recall0.9: 0.1427 - accuracy: 0.8828 - auc: 0
.8423 - f1: 0.2631 - val_loss: 1.2386 - val_tp0.1: 748635.0000 - val_fp0.1: 2823638.0000 - val_tn0.1: 1287705.0000 - val_fn0.1: 55222.0000 - val_precision0.1: 0.2096 - val_recall0.1: 0.931
3 - val_tp0.3: 581617.0000 - val_fp0.3: 1541416.0000 - val_tn0.3: 2569927.0000 - val_fn0.3: 222240.0000 - val_precision0.3: 0.2740 - val_recall0.3: 0.7235 - val_tp0.5: 438469.0000 - val_fp
0.5: 838651.0000 - val_tn0.5: 3272692.0000 - val_fn0.5: 365388.0000 - val_precision0.5: 0.3433 - val_recall0.5: 0.5455 - val_tp0.7: 283136.0000 - val_fp0.7: 250330.0000 - val_tn0.7: 386101
3.0000 - val_fn0.7: 520721.0000 - val_precision0.7: 0.5307 - val_recall0.7: 0.3522 - val_tp0.9: 100313.0000 - val_fp0.9: 12675.0000 - val_tn0.9: 4098668.0000 - val_fn0.9: 703544.0000 - val
_precision0.9: 0.8878 - val_recall0.9: 0.1248 - val_accuracy: 0.7550 - val_auc: 0.7546 - val_f1: 0.2811
Epoch 4/20
800/800 [==============================] - 23s 28ms/step - loss: 0.8790 - tp0.1: 2622979.0000 - fp0.1: 6082638.0000 - tn0.1: 10600330.0000 - fn0.1: 354853.0000 - precision0.1: 0.3013 - rec
all0.1: 0.8808 - tp0.3: 1907681.0000 - fp0.3: 1754807.0000 - tn0.3: 14928161.0000 - fn0.3: 1070151.0000 - precision0.3: 0.5209 - recall0.3: 0.6406 - tp0.5: 1430397.0000 - fp0.5: 638842.000
0 - tn0.5: 16044126.0000 - fn0.5: 1547435.0000 - precision0.5: 0.6913 - recall0.5: 0.4803 - tp0.7: 1005392.0000 - fp0.7: 200266.0000 - tn0.7: 16482702.0000 - fn0.7: 1972440.0000 - precisio
n0.7: 0.8339 - recall0.7: 0.3376 - tp0.9: 537695.0000 - fp0.9: 32470.0000 - tn0.9: 16650498.0000 - fn0.9: 2440137.0000 - precision0.9: 0.9431 - recall0.9: 0.1806 - accuracy: 0.8888 - auc:
0.8539 - f1: 0.2631 - val_loss: 1.4382 - val_tp0.1: 759788.0000 - val_fp0.1: 3033777.0000 - val_tn0.1: 1077566.0000 - val_fn0.1: 44069.0000 - val_precision0.1: 0.2003 - val_recall0.1: 0.94
52 - val_tp0.3: 676790.0000 - val_fp0.3: 2190828.0000 - val_tn0.3: 1920515.0000 - val_fn0.3: 127067.0000 - val_precision0.3: 0.2360 - val_recall0.3: 0.8419 - val_tp0.5: 587381.0000 - val_f
p0.5: 1457790.0000 - val_tn0.5: 2653553.0000 - val_fn0.5: 216476.0000 - val_precision0.5: 0.2872 - val_recall0.5: 0.7307 - val_tp0.7: 467177.0000 - val_fp0.7: 733730.0000 - val_tn0.7: 3377
613.0000 - val_fn0.7: 336680.0000 - val_precision0.7: 0.3890 - val_recall0.7: 0.5812 - val_tp0.9: 265920.0000 - val_fp0.9: 128059.0000 - val_tn0.9: 3983284.0000 - val_fn0.9: 537937.0000 -
val_precision0.9: 0.6750 - val_recall0.9: 0.3308 - val_accuracy: 0.6594 - val_auc: 0.7741 - val_f1: 0.2811
Epoch 5/20
800/800 [==============================] - 23s 29ms/step - loss: 0.8323 - tp0.1: 2632941.0000 - fp0.1: 5640241.0000 - tn0.1: 11042727.0000 - fn0.1: 344891.0000 - precision0.1: 0.3183 - rec
all0.1: 0.8842 - tp0.3: 1987373.0000 - fp0.3: 1723448.0000 - tn0.3: 14959520.0000 - fn0.3: 990459.0000 - precision0.3: 0.5356 - recall0.3: 0.6674 - tp0.5: 1504902.0000 - fp0.5: 660742.0000
 - tn0.5: 16022226.0000 - fn0.5: 1472930.0000 - precision0.5: 0.6949 - recall0.5: 0.5054 - tp0.7: 1062651.0000 - fp0.7: 214795.0000 - tn0.7: 16468173.0000 - fn0.7: 1915181.0000 - precision
0.7: 0.8319 - recall0.7: 0.3569 - tp0.9: 582194.0000 - fp0.9: 33645.0000 - tn0.9: 16649323.0000 - fn0.9: 2395638.0000 - precision0.9: 0.9454 - recall0.9: 0.1955 - accuracy: 0.8915 - auc: 0
.8647 - f1: 0.2631 - val_loss: 1.0148 - val_tp0.1: 708531.0000 - val_fp0.1: 1723364.0000 - val_tn0.1: 2387979.0000 - val_fn0.1: 95326.0000 - val_precision0.1: 0.2913 - val_recall0.1: 0.881
4 - val_tp0.3: 592243.0000 - val_fp0.3: 884014.0000 - val_tn0.3: 3227329.0000 - val_fn0.3: 211614.0000 - val_precision0.3: 0.4012 - val_recall0.3: 0.7368 - val_tp0.5: 463295.0000 - val_fp0
.5: 407207.0000 - val_tn0.5: 3704136.0000 - val_fn0.5: 340562.0000 - val_precision0.5: 0.5322 - val_recall0.5: 0.5763 - val_tp0.7: 324633.0000 - val_fp0.7: 153907.0000 - val_tn0.7: 3957436
.0000 - val_fn0.7: 479224.0000 - val_precision0.7: 0.6784 - val_recall0.7: 0.4038 - val_tp0.9: 171994.0000 - val_fp0.9: 27494.0000 - val_tn0.9: 4083849.0000 - val_fn0.9: 631863.0000 - val_
precision0.9: 0.8622 - val_recall0.9: 0.2140 - val_accuracy: 0.8479 - val_auc: 0.8326 - val_f1: 0.2811
Epoch 6/20
800/800 [==============================] - 23s 29ms/step - loss: 0.8051 - tp0.1: 2656801.0000 - fp0.1: 5609482.0000 - tn0.1: 11073486.0000 - fn0.1: 321031.0000 - precision0.1: 0.3214 - rec
all0.1: 0.8922 - tp0.3: 2004568.0000 - fp0.3: 1721622.0000 - tn0.3: 14961346.0000 - fn0.3: 973264.0000 - precision0.3: 0.5380 - recall0.3: 0.6732 - tp0.5: 1535081.0000 - fp0.5: 636708.0000
 - tn0.5: 16046260.0000 - fn0.5: 1442751.0000 - precision0.5: 0.7068 - recall0.5: 0.5155 - tp0.7: 1104868.0000 - fp0.7: 195727.0000 - tn0.7: 16487241.0000 - fn0.7: 1872964.0000 - precision
0.7: 0.8495 - recall0.7: 0.3710 - tp0.9: 638669.0000 - fp0.9: 30016.0000 - tn0.9: 16652952.0000 - fn0.9: 2339163.0000 - precision0.9: 0.9551 - recall0.9: 0.2145 - accuracy: 0.8942 - auc: 0
.8708 - f1: 0.2631 - val_loss: 1.1897 - val_tp0.1: 687813.0000 - val_fp0.1: 1965439.0000 - val_tn0.1: 2145904.0000 - val_fn0.1: 116044.0000 - val_precision0.1: 0.2592 - val_recall0.1: 0.85
56 - val_tp0.3: 589213.0000 - val_fp0.3: 1346718.0000 - val_tn0.3: 2764625.0000 - val_fn0.3: 214644.0000 - val_precision0.3: 0.3044 - val_recall0.3: 0.7330 - val_tp0.5: 480400.0000 - val_f
p0.5: 797605.0000 - val_tn0.5: 3313738.0000 - val_fn0.5: 323457.0000 - val_precision0.5: 0.3759 - val_recall0.5: 0.5976 - val_tp0.7: 356338.0000 - val_fp0.7: 349313.0000 - val_tn0.7: 37620
30.0000 - val_fn0.7: 447519.0000 - val_precision0.7: 0.5050 - val_recall0.7: 0.4433 - val_tp0.9: 203593.0000 - val_fp0.9: 63510.0000 - val_tn0.9: 4047833.0000 - val_fn0.9: 600264.0000 - va
l_precision0.9: 0.7622 - val_recall0.9: 0.2533 - val_accuracy: 0.7719 - val_auc: 0.7799 - val_f1: 0.2811
Epoch 7/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7730 - tp0.1: 2661167.0000 - fp0.1: 5252891.0000 - tn0.1: 11430077.0000 - fn0.1: 316665.0000 - precision0.1: 0.3363 - rec
all0.1: 0.8937 - tp0.3: 2045078.0000 - fp0.3: 1661108.0000 - tn0.3: 15021860.0000 - fn0.3: 932754.0000 - precision0.3: 0.5518 - recall0.3: 0.6868 - tp0.5: 1572828.0000 - fp0.5: 627634.0000
 - tn0.5: 16055334.0000 - fn0.5: 1405004.0000 - precision0.5: 0.7148 - recall0.5: 0.5282 - tp0.7: 1148247.0000 - fp0.7: 206720.0000 - tn0.7: 16476248.0000 - fn0.7: 1829585.0000 - precision
0.7: 0.8474 - recall0.7: 0.3856 - tp0.9: 702777.0000 - fp0.9: 37332.0000 - tn0.9: 16645636.0000 - fn0.9: 2275055.0000 - precision0.9: 0.9496 - recall0.9: 0.2360 - accuracy: 0.8966 - auc: 0
.8779 - f1: 0.2631 - val_loss: 0.9848 - val_tp0.1: 705842.0000 - val_fp0.1: 1730539.0000 - val_tn0.1: 2380804.0000 - val_fn0.1: 98015.0000 - val_precision0.1: 0.2897 - val_recall0.1: 0.878
1 - val_tp0.3: 495957.0000 - val_fp0.3: 601184.0000 - val_tn0.3: 3510159.0000 - val_fn0.3: 307900.0000 - val_precision0.3: 0.4520 - val_recall0.3: 0.6170 - val_tp0.5: 353962.0000 - val_fp0
.5: 226006.0000 - val_tn0.5: 3885337.0000 - val_fn0.5: 449895.0000 - val_precision0.5: 0.6103 - val_recall0.5: 0.4403 - val_tp0.7: 232086.0000 - val_fp0.7: 57808.0000 - val_tn0.7: 4053535.
0000 - val_fn0.7: 571771.0000 - val_precision0.7: 0.8006 - val_recall0.7: 0.2887 - val_tp0.9: 109899.0000 - val_fp0.9: 4377.0000 - val_tn0.9: 4106966.0000 - val_fn0.9: 693958.0000 - val_pr
ecision0.9: 0.9617 - val_recall0.9: 0.1367 - val_accuracy: 0.8625 - val_auc: 0.8220 - val_f1: 0.2811
Epoch 8/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7537 - tp0.1: 2668427.0000 - fp0.1: 5092328.0000 - tn0.1: 11590640.0000 - fn0.1: 309405.0000 - precision0.1: 0.3438 - rec
all0.1: 0.8961 - tp0.3: 2091292.0000 - fp0.3: 1655195.0000 - tn0.3: 15027773.0000 - fn0.3: 886540.0000 - precision0.3: 0.5582 - recall0.3: 0.7023 - tp0.5: 1591876.0000 - fp0.5: 604215.0000
 - tn0.5: 16078753.0000 - fn0.5: 1385956.0000 - precision0.5: 0.7249 - recall0.5: 0.5346 - tp0.7: 1154686.0000 - fp0.7: 198461.0000 - tn0.7: 16484507.0000 - fn0.7: 1823146.0000 - precision
0.7: 0.8533 - recall0.7: 0.3878 - tp0.9: 706986.0000 - fp0.9: 36859.0000 - tn0.9: 16646109.0000 - fn0.9: 2270846.0000 - precision0.9: 0.9504 - recall0.9: 0.2374 - accuracy: 0.8988 - auc: 0
.8827 - f1: 0.2631 - val_loss: 1.0536 - val_tp0.1: 717248.0000 - val_fp0.1: 2127277.0000 - val_tn0.1: 1984066.0000 - val_fn0.1: 86609.0000 - val_precision0.1: 0.2522 - val_recall0.1: 0.892
3 - val_tp0.3: 516820.0000 - val_fp0.3: 718092.0000 - val_tn0.3: 3393251.0000 - val_fn0.3: 287037.0000 - val_precision0.3: 0.4185 - val_recall0.3: 0.6429 - val_tp0.5: 349749.0000 - val_fp0
.5: 209234.0000 - val_tn0.5: 3902109.0000 - val_fn0.5: 454108.0000 - val_precision0.5: 0.6257 - val_recall0.5: 0.4351 - val_tp0.7: 217539.0000 - val_fp0.7: 55114.0000 - val_tn0.7: 4056229.
0000 - val_fn0.7: 586318.0000 - val_precision0.7: 0.7979 - val_recall0.7: 0.2706 - val_tp0.9: 108545.0000 - val_fp0.9: 6637.0000 - val_tn0.9: 4104706.0000 - val_fn0.9: 695312.0000 - val_pr
ecision0.9: 0.9424 - val_recall0.9: 0.1350 - val_accuracy: 0.8650 - val_auc: 0.8110 - val_f1: 0.2811
Epoch 9/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7245 - tp0.1: 2670184.0000 - fp0.1: 4736653.0000 - tn0.1: 11946315.0000 - fn0.1: 307648.0000 - precision0.1: 0.3605 - rec
all0.1: 0.8967 - tp0.3: 2138051.0000 - fp0.3: 1655069.0000 - tn0.3: 15027899.0000 - fn0.3: 839781.0000 - precision0.3: 0.5637 - recall0.3: 0.7180 - tp0.5: 1658889.0000 - fp0.5: 602716.0000
 - tn0.5: 16080252.0000 - fn0.5: 1318943.0000 - precision0.5: 0.7335 - recall0.5: 0.5571 - tp0.7: 1231743.0000 - fp0.7: 199604.0000 - tn0.7: 16483364.0000 - fn0.7: 1746089.0000 - precision
0.7: 0.8605 - recall0.7: 0.4136 - tp0.9: 770284.0000 - fp0.9: 38798.0000 - tn0.9: 16644170.0000 - fn0.9: 2207548.0000 - precision0.9: 0.9520 - recall0.9: 0.2587 - accuracy: 0.9023 - auc: 0
.8890 - f1: 0.2631 - val_loss: 0.9833 - val_tp0.1: 684923.0000 - val_fp0.1: 1482632.0000 - val_tn0.1: 2628711.0000 - val_fn0.1: 118934.0000 - val_precision0.1: 0.3160 - val_recall0.1: 0.85
20 - val_tp0.3: 378339.0000 - val_fp0.3: 205771.0000 - val_tn0.3: 3905572.0000 - val_fn0.3: 425518.0000 - val_precision0.3: 0.6477 - val_recall0.3: 0.4707 - val_tp0.5: 214088.0000 - val_fp
0.5: 39231.0000 - val_tn0.5: 4072112.0000 - val_fn0.5: 589769.0000 - val_precision0.5: 0.8451 - val_recall0.5: 0.2663 - val_tp0.7: 124578.0000 - val_fp0.7: 7623.0000 - val_tn0.7: 4103720.0
000 - val_fn0.7: 679279.0000 - val_precision0.7: 0.9423 - val_recall0.7: 0.1550 - val_tp0.9: 59851.0000 - val_fp0.9: 608.0000 - val_tn0.9: 4110735.0000 - val_fn0.9: 744006.0000 - val_preci
sion0.9: 0.9899 - val_recall0.9: 0.0745 - val_accuracy: 0.8720 - val_auc: 0.8270 - val_f1: 0.2811
Epoch 10/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7278 - tp0.1: 2665983.0000 - fp0.1: 4770602.0000 - tn0.1: 11912366.0000 - fn0.1: 311849.0000 - precision0.1: 0.3585 - rec
all0.1: 0.8953 - tp0.3: 2137054.0000 - fp0.3: 1618394.0000 - tn0.3: 15064574.0000 - fn0.3: 840778.0000 - precision0.3: 0.5691 - recall0.3: 0.7177 - tp0.5: 1657865.0000 - fp0.5: 592475.0000
 - tn0.5: 16090493.0000 - fn0.5: 1319967.0000 - precision0.5: 0.7367 - recall0.5: 0.5567 - tp0.7: 1234306.0000 - fp0.7: 193079.0000 - tn0.7: 16489889.0000 - fn0.7: 1743526.0000 - precision
0.7: 0.8647 - recall0.7: 0.4145 - tp0.9: 773361.0000 - fp0.9: 37289.0000 - tn0.9: 16645679.0000 - fn0.9: 2204471.0000 - precision0.9: 0.9540 - recall0.9: 0.2597 - accuracy: 0.9027 - auc: 0
.8886 - f1: 0.2631 - val_loss: 0.9507 - val_tp0.1: 693260.0000 - val_fp0.1: 1463538.0000 - val_tn0.1: 2647805.0000 - val_fn0.1: 110597.0000 - val_precision0.1: 0.3214 - val_recall0.1: 0.86
24 - val_tp0.3: 398366.0000 - val_fp0.3: 212804.0000 - val_tn0.3: 3898539.0000 - val_fn0.3: 405491.0000 - val_precision0.3: 0.6518 - val_recall0.3: 0.4956 - val_tp0.5: 241139.0000 - val_fp
0.5: 41379.0000 - val_tn0.5: 4069964.0000 - val_fn0.5: 562718.0000 - val_precision0.5: 0.8535 - val_recall0.5: 0.3000 - val_tp0.7: 143798.0000 - val_fp0.7: 8441.0000 - val_tn0.7: 4102902.0
000 - val_fn0.7: 660059.0000 - val_precision0.7: 0.9446 - val_recall0.7: 0.1789 - val_tp0.9: 73993.0000 - val_fp0.9: 945.0000 - val_tn0.9: 4110398.0000 - val_fn0.9: 729864.0000 - val_preci
sion0.9: 0.9874 - val_recall0.9: 0.0920 - val_accuracy: 0.8771 - val_auc: 0.8421 - val_f1: 0.2811
Epoch 11/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7098 - tp0.1: 2671173.0000 - fp0.1: 4583226.0000 - tn0.1: 12099742.0000 - fn0.1: 306659.0000 - precision0.1: 0.3682 - rec
all0.1: 0.8970 - tp0.3: 2161831.0000 - fp0.3: 1611405.0000 - tn0.3: 15071563.0000 - fn0.3: 816001.0000 - precision0.3: 0.5729 - recall0.3: 0.7260 - tp0.5: 1683781.0000 - fp0.5: 593951.0000
 - tn0.5: 16089017.0000 - fn0.5: 1294051.0000 - precision0.5: 0.7392 - recall0.5: 0.5654 - tp0.7: 1246509.0000 - fp0.7: 197501.0000 - tn0.7: 16485467.0000 - fn0.7: 1731323.0000 - precision
0.7: 0.8632 - recall0.7: 0.4186 - tp0.9: 780021.0000 - fp0.9: 37187.0000 - tn0.9: 16645781.0000 - fn0.9: 2197811.0000 - precision0.9: 0.9545 - recall0.9: 0.2619 - accuracy: 0.9040 - auc: 0
.8921 - f1: 0.2631 - val_loss: 0.9881 - val_tp0.1: 684864.0000 - val_fp0.1: 1514096.0000 - val_tn0.1: 2597247.0000 - val_fn0.1: 118993.0000 - val_precision0.1: 0.3114 - val_recall0.1: 0.85
20 - val_tp0.3: 448578.0000 - val_fp0.3: 437347.0000 - val_tn0.3: 3673996.0000 - val_fn0.3: 355279.0000 - val_precision0.3: 0.5063 - val_recall0.3: 0.5580 - val_tp0.5: 306480.0000 - val_fp
0.5: 142586.0000 - val_tn0.5: 3968757.0000 - val_fn0.5: 497377.0000 - val_precision0.5: 0.6825 - val_recall0.5: 0.3813 - val_tp0.7: 196534.0000 - val_fp0.7: 34758.0000 - val_tn0.7: 4076585
.0000 - val_fn0.7: 607323.0000 - val_precision0.7: 0.8497 - val_recall0.7: 0.2445 - val_tp0.9: 93543.0000 - val_fp0.9: 2618.0000 - val_tn0.9: 4108725.0000 - val_fn0.9: 710314.0000 - val_pr
ecision0.9: 0.9728 - val_recall0.9: 0.1164 - val_accuracy: 0.8698 - val_auc: 0.8226 - val_f1: 0.2811
Epoch 12/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7115 - tp0.1: 2672752.0000 - fp0.1: 4630371.0000 - tn0.1: 12052597.0000 - fn0.1: 305080.0000 - precision0.1: 0.3660 - rec
all0.1: 0.8975 - tp0.3: 2159193.0000 - fp0.3: 1616151.0000 - tn0.3: 15066817.0000 - fn0.3: 818639.0000 - precision0.3: 0.5719 - recall0.3: 0.7251 - tp0.5: 1689796.0000 - fp0.5: 588788.0000
 - tn0.5: 16094180.0000 - fn0.5: 1288036.0000 - precision0.5: 0.7416 - recall0.5: 0.5675 - tp0.7: 1257203.0000 - fp0.7: 187802.0000 - tn0.7: 16495166.0000 - fn0.7: 1720629.0000 - precision
0.7: 0.8700 - recall0.7: 0.4222 - tp0.9: 781350.0000 - fp0.9: 35537.0000 - tn0.9: 16647431.0000 - fn0.9: 2196482.0000 - precision0.9: 0.9565 - recall0.9: 0.2624 - accuracy: 0.9045 - auc: 0
.8920 - f1: 0.2631 - val_loss: 0.9426 - val_tp0.1: 694538.0000 - val_fp0.1: 1496227.0000 - val_tn0.1: 2615116.0000 - val_fn0.1: 109319.0000 - val_precision0.1: 0.3170 - val_recall0.1: 0.86
40 - val_tp0.3: 506023.0000 - val_fp0.3: 520642.0000 - val_tn0.3: 3590701.0000 - val_fn0.3: 297834.0000 - val_precision0.3: 0.4929 - val_recall0.3: 0.6295 - val_tp0.5: 346422.0000 - val_fp
0.5: 153015.0000 - val_tn0.5: 3958328.0000 - val_fn0.5: 457435.0000 - val_precision0.5: 0.6936 - val_recall0.5: 0.4309 - val_tp0.7: 221842.0000 - val_fp0.7: 37879.0000 - val_tn0.7: 4073464
.0000 - val_fn0.7: 582015.0000 - val_precision0.7: 0.8542 - val_recall0.7: 0.2760 - val_tp0.9: 104244.0000 - val_fp0.9: 3502.0000 - val_tn0.9: 4107841.0000 - val_fn0.9: 699613.0000 - val_p
recision0.9: 0.9675 - val_recall0.9: 0.1297 - val_accuracy: 0.8758 - val_auc: 0.8365 - val_f1: 0.2811
Epoch 13/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7049 - tp0.1: 2669030.0000 - fp0.1: 4503237.0000 - tn0.1: 12179731.0000 - fn0.1: 308802.0000 - precision0.1: 0.3721 - rec
all0.1: 0.8963 - tp0.3: 2190933.0000 - fp0.3: 1671712.0000 - tn0.3: 15011256.0000 - fn0.3: 786899.0000 - precision0.3: 0.5672 - recall0.3: 0.7357 - tp0.5: 1721545.0000 - fp0.5: 613878.0000
 - tn0.5: 16069090.0000 - fn0.5: 1256287.0000 - precision0.5: 0.7371 - recall0.5: 0.5781 - tp0.7: 1280739.0000 - fp0.7: 198223.0000 - tn0.7: 16484745.0000 - fn0.7: 1697093.0000 - precision
0.7: 0.8660 - recall0.7: 0.4301 - tp0.9: 795579.0000 - fp0.9: 35764.0000 - tn0.9: 16647204.0000 - fn0.9: 2182253.0000 - precision0.9: 0.9570 - recall0.9: 0.2672 - accuracy: 0.9049 - auc: 0
.8932 - f1: 0.2631 - val_loss: 0.9456 - val_tp0.1: 699033.0000 - val_fp0.1: 1563859.0000 - val_tn0.1: 2547484.0000 - val_fn0.1: 104824.0000 - val_precision0.1: 0.3089 - val_recall0.1: 0.86
96 - val_tp0.3: 540655.0000 - val_fp0.3: 610467.0000 - val_tn0.3: 3500876.0000 - val_fn0.3: 263202.0000 - val_precision0.3: 0.4697 - val_recall0.3: 0.6726 - val_tp0.5: 379380.0000 - val_fp
0.5: 190187.0000 - val_tn0.5: 3921156.0000 - val_fn0.5: 424477.0000 - val_precision0.5: 0.6661 - val_recall0.5: 0.4719 - val_tp0.7: 250875.0000 - val_fp0.7: 56341.0000 - val_tn0.7: 4055002
.0000 - val_fn0.7: 552982.0000 - val_precision0.7: 0.8166 - val_recall0.7: 0.3121 - val_tp0.9: 123390.0000 - val_fp0.9: 6935.0000 - val_tn0.9: 4104408.0000 - val_fn0.9: 680467.0000 - val_p
recision0.9: 0.9468 - val_recall0.9: 0.1535 - val_accuracy: 0.8749 - val_auc: 0.8387 - val_f1: 0.2811
Epoch 14/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7025 - tp0.1: 2675299.0000 - fp0.1: 4563270.0000 - tn0.1: 12119698.0000 - fn0.1: 302533.0000 - precision0.1: 0.3696 - rec
all0.1: 0.8984 - tp0.3: 2177828.0000 - fp0.3: 1603216.0000 - tn0.3: 15079752.0000 - fn0.3: 800004.0000 - precision0.3: 0.5760 - recall0.3: 0.7313 - tp0.5: 1706323.0000 - fp0.5: 592574.0000
 - tn0.5: 16090394.0000 - fn0.5: 1271509.0000 - precision0.5: 0.7422 - recall0.5: 0.5730 - tp0.7: 1267052.0000 - fp0.7: 191517.0000 - tn0.7: 16491451.0000 - fn0.7: 1710780.0000 - precision
0.7: 0.8687 - recall0.7: 0.4255 - tp0.9: 779803.0000 - fp0.9: 33525.0000 - tn0.9: 16649443.0000 - fn0.9: 2198029.0000 - precision0.9: 0.9588 - recall0.9: 0.2619 - accuracy: 0.9052 - auc: 0
.8942 - f1: 0.2631 - val_loss: 0.9513 - val_tp0.1: 685976.0000 - val_fp0.1: 1445496.0000 - val_tn0.1: 2665847.0000 - val_fn0.1: 117881.0000 - val_precision0.1: 0.3218 - val_recall0.1: 0.85
34 - val_tp0.3: 502057.0000 - val_fp0.3: 484827.0000 - val_tn0.3: 3626516.0000 - val_fn0.3: 301800.0000 - val_precision0.3: 0.5087 - val_recall0.3: 0.6246 - val_tp0.5: 350620.0000 - val_fp
0.5: 151636.0000 - val_tn0.5: 3959707.0000 - val_fn0.5: 453237.0000 - val_precision0.5: 0.6981 - val_recall0.5: 0.4362 - val_tp0.7: 231119.0000 - val_fp0.7: 44841.0000 - val_tn0.7: 4066502
.0000 - val_fn0.7: 572738.0000 - val_precision0.7: 0.8375 - val_recall0.7: 0.2875 - val_tp0.9: 117775.0000 - val_fp0.9: 6325.0000 - val_tn0.9: 4105018.0000 - val_fn0.9: 686082.0000 - val_p
recision0.9: 0.9490 - val_recall0.9: 0.1465 - val_accuracy: 0.8769 - val_auc: 0.8355 - val_f1: 0.2811
Epoch 15/20
800/800 [==============================] - 23s 29ms/step - loss: 0.6970 - tp0.1: 2674241.0000 - fp0.1: 4479713.0000 - tn0.1: 12203255.0000 - fn0.1: 303591.0000 - precision0.1: 0.3738 - rec
all0.1: 0.8980 - tp0.3: 2170500.0000 - fp0.3: 1588775.0000 - tn0.3: 15094193.0000 - fn0.3: 807332.0000 - precision0.3: 0.5774 - recall0.3: 0.7289 - tp0.5: 1716769.0000 - fp0.5: 596411.0000
 - tn0.5: 16086557.0000 - fn0.5: 1261063.0000 - precision0.5: 0.7422 - recall0.5: 0.5765 - tp0.7: 1295621.0000 - fp0.7: 194669.0000 - tn0.7: 16488299.0000 - fn0.7: 1682211.0000 - precision
0.7: 0.8694 - recall0.7: 0.4351 - tp0.9: 814054.0000 - fp0.9: 35899.0000 - tn0.9: 16647069.0000 - fn0.9: 2163778.0000 - precision0.9: 0.9578 - recall0.9: 0.2734 - accuracy: 0.9055 - auc: 0
.8945 - f1: 0.2631 - val_loss: 0.9517 - val_tp0.1: 688515.0000 - val_fp0.1: 1481139.0000 - val_tn0.1: 2630204.0000 - val_fn0.1: 115342.0000 - val_precision0.1: 0.3173 - val_recall0.1: 0.85
65 - val_tp0.3: 506409.0000 - val_fp0.3: 477938.0000 - val_tn0.3: 3633405.0000 - val_fn0.3: 297448.0000 - val_precision0.3: 0.5145 - val_recall0.3: 0.6300 - val_tp0.5: 352279.0000 - val_fp
0.5: 150937.0000 - val_tn0.5: 3960406.0000 - val_fn0.5: 451578.0000 - val_precision0.5: 0.7001 - val_recall0.5: 0.4382 - val_tp0.7: 231093.0000 - val_fp0.7: 45551.0000 - val_tn0.7: 4065792
.0000 - val_fn0.7: 572764.0000 - val_precision0.7: 0.8353 - val_recall0.7: 0.2875 - val_tp0.9: 118279.0000 - val_fp0.9: 6604.0000 - val_tn0.9: 4104739.0000 - val_fn0.9: 685578.0000 - val_p
recision0.9: 0.9471 - val_recall0.9: 0.1471 - val_accuracy: 0.8774 - val_auc: 0.8370 - val_f1: 0.2811
Epoch 16/20
800/800 [==============================] - 23s 29ms/step - loss: 0.6972 - tp0.1: 2674892.0000 - fp0.1: 4478986.0000 - tn0.1: 12203982.0000 - fn0.1: 302940.0000 - precision0.1: 0.3739 - rec
all0.1: 0.8983 - tp0.3: 2177848.0000 - fp0.3: 1577462.0000 - tn0.3: 15105506.0000 - fn0.3: 799984.0000 - precision0.3: 0.5799 - recall0.3: 0.7314 - tp0.5: 1710580.0000 - fp0.5: 588844.0000
 - tn0.5: 16094124.0000 - fn0.5: 1267252.0000 - precision0.5: 0.7439 - recall0.5: 0.5744 - tp0.7: 1276251.0000 - fp0.7: 191455.0000 - tn0.7: 16491513.0000 - fn0.7: 1701581.0000 - precision
0.7: 0.8696 - recall0.7: 0.4286 - tp0.9: 794990.0000 - fp0.9: 33049.0000 - tn0.9: 16649919.0000 - fn0.9: 2182842.0000 - precision0.9: 0.9601 - recall0.9: 0.2670 - accuracy: 0.9056 - auc: 0
.8951 - f1: 0.2631 - val_loss: 0.9505 - val_tp0.1: 684484.0000 - val_fp0.1: 1423827.0000 - val_tn0.1: 2687516.0000 - val_fn0.1: 119373.0000 - val_precision0.1: 0.3247 - val_recall0.1: 0.85
15 - val_tp0.3: 507858.0000 - val_fp0.3: 511252.0000 - val_tn0.3: 3600091.0000 - val_fn0.3: 295999.0000 - val_precision0.3: 0.4983 - val_recall0.3: 0.6318 - val_tp0.5: 356641.0000 - val_fp
0.5: 160219.0000 - val_tn0.5: 3951124.0000 - val_fn0.5: 447216.0000 - val_precision0.5: 0.6900 - val_recall0.5: 0.4437 - val_tp0.7: 236723.0000 - val_fp0.7: 47470.0000 - val_tn0.7: 4063873
.0000 - val_fn0.7: 567134.0000 - val_precision0.7: 0.8330 - val_recall0.7: 0.2945 - val_tp0.9: 119706.0000 - val_fp0.9: 6487.0000 - val_tn0.9: 4104856.0000 - val_fn0.9: 684151.0000 - val_p
recision0.9: 0.9486 - val_recall0.9: 0.1489 - val_accuracy: 0.8764 - val_auc: 0.8350 - val_f1: 0.2811
Epoch 17/20
800/800 [==============================] - 23s 29ms/step - loss: 0.6981 - tp0.1: 2674601.0000 - fp0.1: 4489171.0000 - tn0.1: 12193797.0000 - fn0.1: 303231.0000 - precision0.1: 0.3734 - rec
all0.1: 0.8982 - tp0.3: 2183675.0000 - fp0.3: 1573762.0000 - tn0.3: 15109206.0000 - fn0.3: 794157.0000 - precision0.3: 0.5812 - recall0.3: 0.7333 - tp0.5: 1713199.0000 - fp0.5: 586718.0000
 - tn0.5: 16096250.0000 - fn0.5: 1264633.0000 - precision0.5: 0.7449 - recall0.5: 0.5753 - tp0.7: 1283634.0000 - fp0.7: 195576.0000 - tn0.7: 16487392.0000 - fn0.7: 1694198.0000 - precision
0.7: 0.8678 - recall0.7: 0.4311 - tp0.9: 805115.0000 - fp0.9: 36004.0000 - tn0.9: 16646964.0000 - fn0.9: 2172717.0000 - precision0.9: 0.9572 - recall0.9: 0.2704 - accuracy: 0.9058 - auc: 0
.8950 - f1: 0.2631 - val_loss: 0.9465 - val_tp0.1: 687031.0000 - val_fp0.1: 1439264.0000 - val_tn0.1: 2672079.0000 - val_fn0.1: 116826.0000 - val_precision0.1: 0.3231 - val_recall0.1: 0.85
47 - val_tp0.3: 508441.0000 - val_fp0.3: 510821.0000 - val_tn0.3: 3600522.0000 - val_fn0.3: 295416.0000 - val_precision0.3: 0.4988 - val_recall0.3: 0.6325 - val_tp0.5: 352823.0000 - val_fp
0.5: 152954.0000 - val_tn0.5: 3958389.0000 - val_fn0.5: 451034.0000 - val_precision0.5: 0.6976 - val_recall0.5: 0.4389 - val_tp0.7: 231208.0000 - val_fp0.7: 43355.0000 - val_tn0.7: 4067988
.0000 - val_fn0.7: 572649.0000 - val_precision0.7: 0.8421 - val_recall0.7: 0.2876 - val_tp0.9: 114713.0000 - val_fp0.9: 5450.0000 - val_tn0.9: 4105893.0000 - val_fn0.9: 689144.0000 - val_p
recision0.9: 0.9546 - val_recall0.9: 0.1427 - val_accuracy: 0.8771 - val_auc: 0.8361 - val_f1: 0.2811
Epoch 18/20
800/800 [==============================] - 23s 29ms/step - loss: 0.6969 - tp0.1: 2676617.0000 - fp0.1: 4470158.0000 - tn0.1: 12212810.0000 - fn0.1: 301215.0000 - precision0.1: 0.3745 - rec
all0.1: 0.8988 - tp0.3: 2177086.0000 - fp0.3: 1601828.0000 - tn0.3: 15081140.0000 - fn0.3: 800746.0000 - precision0.3: 0.5761 - recall0.3: 0.7311 - tp0.5: 1715470.0000 - fp0.5: 611086.0000
 - tn0.5: 16071882.0000 - fn0.5: 1262362.0000 - precision0.5: 0.7373 - recall0.5: 0.5761 - tp0.7: 1288609.0000 - fp0.7: 198192.0000 - tn0.7: 16484776.0000 - fn0.7: 1689223.0000 - precision
0.7: 0.8667 - recall0.7: 0.4327 - tp0.9: 807642.0000 - fp0.9: 37272.0000 - tn0.9: 16645696.0000 - fn0.9: 2170190.0000 - precision0.9: 0.9559 - recall0.9: 0.2712 - accuracy: 0.9047 - auc: 0
.8948 - f1: 0.2631 - val_loss: 0.9503 - val_tp0.1: 684845.0000 - val_fp0.1: 1434583.0000 - val_tn0.1: 2676760.0000 - val_fn0.1: 119012.0000 - val_precision0.1: 0.3231 - val_recall0.1: 0.85
19 - val_tp0.3: 501142.0000 - val_fp0.3: 471764.0000 - val_tn0.3: 3639579.0000 - val_fn0.3: 302715.0000 - val_precision0.3: 0.5151 - val_recall0.3: 0.6234 - val_tp0.5: 347005.0000 - val_fp
0.5: 143102.0000 - val_tn0.5: 3968241.0000 - val_fn0.5: 456852.0000 - val_precision0.5: 0.7080 - val_recall0.5: 0.4317 - val_tp0.7: 226199.0000 - val_fp0.7: 40947.0000 - val_tn0.7: 4070396
.0000 - val_fn0.7: 577658.0000 - val_precision0.7: 0.8467 - val_recall0.7: 0.2814 - val_tp0.9: 113524.0000 - val_fp0.9: 5443.0000 - val_tn0.9: 4105900.0000 - val_fn0.9: 690333.0000 - val_p
recision0.9: 0.9542 - val_recall0.9: 0.1412 - val_accuracy: 0.8779 - val_auc: 0.8363 - val_f1: 0.2811
Epoch 19/20
800/800 [==============================] - 23s 28ms/step - loss: 0.6974 - tp0.1: 2678075.0000 - fp0.1: 4501081.0000 - tn0.1: 12181887.0000 - fn0.1: 299757.0000 - precision0.1: 0.3730 - rec
all0.1: 0.8993 - tp0.3: 2193569.0000 - fp0.3: 1620047.0000 - tn0.3: 15062921.0000 - fn0.3: 784263.0000 - precision0.3: 0.5752 - recall0.3: 0.7366 - tp0.5: 1721203.0000 - fp0.5: 604162.0000
 - tn0.5: 16078806.0000 - fn0.5: 1256629.0000 - precision0.5: 0.7402 - recall0.5: 0.5780 - tp0.7: 1284180.0000 - fp0.7: 199088.0000 - tn0.7: 16483880.0000 - fn0.7: 1693652.0000 - precision
0.7: 0.8658 - recall0.7: 0.4312 - tp0.9: 797069.0000 - fp0.9: 36223.0000 - tn0.9: 16646745.0000 - fn0.9: 2180763.0000 - precision0.9: 0.9565 - recall0.9: 0.2677 - accuracy: 0.9054 - auc: 0
.8954 - f1: 0.2631 - val_loss: 0.9439 - val_tp0.1: 694044.0000 - val_fp0.1: 1504048.0000 - val_tn0.1: 2607295.0000 - val_fn0.1: 109813.0000 - val_precision0.1: 0.3157 - val_recall0.1: 0.86
34 - val_tp0.3: 523881.0000 - val_fp0.3: 545389.0000 - val_tn0.3: 3565954.0000 - val_fn0.3: 279976.0000 - val_precision0.3: 0.4899 - val_recall0.3: 0.6517 - val_tp0.5: 369154.0000 - val_fp
0.5: 174373.0000 - val_tn0.5: 3936970.0000 - val_fn0.5: 434703.0000 - val_precision0.5: 0.6792 - val_recall0.5: 0.4592 - val_tp0.7: 245997.0000 - val_fp0.7: 52312.0000 - val_tn0.7: 4059031
.0000 - val_fn0.7: 557860.0000 - val_precision0.7: 0.8246 - val_recall0.7: 0.3060 - val_tp0.9: 124029.0000 - val_fp0.9: 7043.0000 - val_tn0.9: 4104300.0000 - val_fn0.9: 679828.0000 - val_p
recision0.9: 0.9463 - val_recall0.9: 0.1543 - val_accuracy: 0.8761 - val_auc: 0.8387 - val_f1: 0.2811
Epoch 20/20
800/800 [==============================] - 23s 29ms/step - loss: 0.7034 - tp0.1: 2668995.0000 - fp0.1: 4472643.0000 - tn0.1: 12210325.0000 - fn0.1: 308837.0000 - precision0.1: 0.3737 - rec
all0.1: 0.8963 - tp0.3: 2164194.0000 - fp0.3: 1572121.0000 - tn0.3: 15110847.0000 - fn0.3: 813638.0000 - precision0.3: 0.5792 - recall0.3: 0.7268 - tp0.5: 1680381.0000 - fp0.5: 579083.0000
 - tn0.5: 16103885.0000 - fn0.5: 1297451.0000 - precision0.5: 0.7437 - recall0.5: 0.5643 - tp0.7: 1241011.0000 - fp0.7: 190475.0000 - tn0.7: 16492493.0000 - fn0.7: 1736821.0000 - precision
0.7: 0.8669 - recall0.7: 0.4167 - tp0.9: 785981.0000 - fp0.9: 34923.0000 - tn0.9: 16648045.0000 - fn0.9: 2191851.0000 - precision0.9: 0.9575 - recall0.9: 0.2639 - accuracy: 0.9046 - auc: 0
.8932 - f1: 0.2631 - val_loss: 0.9520 - val_tp0.1: 682980.0000 - val_fp0.1: 1413218.0000 - val_tn0.1: 2698125.0000 - val_fn0.1: 120877.0000 - val_precision0.1: 0.3258 - val_recall0.1: 0.84
96 - val_tp0.3: 500974.0000 - val_fp0.3: 498780.0000 - val_tn0.3: 3612563.0000 - val_fn0.3: 302883.0000 - val_precision0.3: 0.5011 - val_recall0.3: 0.6232 - val_tp0.5: 348018.0000 - val_fp
0.5: 150788.0000 - val_tn0.5: 3960555.0000 - val_fn0.5: 455839.0000 - val_precision0.5: 0.6977 - val_recall0.5: 0.4329 - val_tp0.7: 228527.0000 - val_fp0.7: 42760.0000 - val_tn0.7: 4068583
.0000 - val_fn0.7: 575330.0000 - val_precision0.7: 0.8424 - val_recall0.7: 0.2843 - val_tp0.9: 113851.0000 - val_fp0.9: 5410.0000 - val_tn0.9: 4105933.0000 - val_fn0.9: 690006.0000 - val_p
recision0.9: 0.9546 - val_recall0.9: 0.1416 - val_accuracy: 0.8766 - val_auc: 0.8340 - val_f1: 0.2811
--- Running training session 60/140
{'hp_epochs': 20, 'hp_batch_size': 6, 'hp_scaler': 'maxabs', 'hp_n_levels': 3, 'hp_first_filters': 16, 'hp_pool_size': 2, 'hp_input_size': 4096, 'hp_lr_start': 0.06810286704342322, 'hp_lr_
power': 5.0}
--- repeat #: 2
input - shape:   (None, 4096, 1)
output - shape:  (None, 4096, 1)
Epoch 1/20
800/800 [==============================] - 41s 33ms/step - loss: 1.1728 - tp0.1: 2643619.0000 - fp0.1: 9398198.0000 - tn0.1: 7284770.0000 - fn0.1: 334213.0000 - precision0.1: 0.2195 - reca
ll0.1: 0.8878 - tp0.3: 1267730.0000 - fp0.3: 3106478.0000 - tn0.3: 13576490.0000 - fn0.3: 1710102.0000 - precision0.3: 0.2898 - recall0.3: 0.4257 - tp0.5: 110715.0000 - fp0.5: 152305.0000
- tn0.5: 16530663.0000 - fn0.5: 2867117.0000 - precision0.5: 0.4209 - recall0.5: 0.0372 - tp0.7: 22534.0000 - fp0.7: 27016.0000 - tn0.7: 16655952.0000 - fn0.7: 2955298.0000 - precision0.7:
 0.4548 - recall0.7: 0.0076 - tp0.9: 5370.0000 - fp0.9: 4276.0000 - tn0.9: 16678692.0000 - fn0.9: 2972462.0000 - precision0.9: 0.5567 - recall0.9: 0.0018 - accuracy: 0.8464 - auc: 0.7210 -
 f1: 0.2631 - val_loss: 1.3911 - val_tp0.1: 803682.0000 - val_fp0.1: 4109861.0000 - val_tn0.1: 1482.0000 - val_fn0.1: 175.0000 - val_precision0.1: 0.1636 - val_recall0.1: 0.9998 - val_tp0.
3: 481990.0000 - val_fp0.3: 1381226.0000 - val_tn0.3: 2730117.0000 - val_fn0.3: 321867.0000 - val_precision0.3: 0.2587 - val_recall0.3: 0.5996 - val_tp0.5: 173457.0000 - val_fp0.5: 387503.
0000 - val_tn0.5: 3723840.0000 - val_fn0.5: 630400.0000 - val_precision0.5: 0.3092 - val_recall0.5: 0.2158 - val_tp0.7: 62452.0000 - val_fp0.7: 51611.0000 - val_tn0.7: 4059732.0000 - val_f
n0.7: 741405.0000 - val_precision0.7: 0.5475 - val_recall0.7: 0.0777 - val_tp0.9: 36.0000 - val_fp0.9: 0.0000e+00 - val_tn0.9: 4111343.0000 - val_fn0.9: 803821.0000 - val_precision0.9: 1.0
000 - val_recall0.9: 4.4784e-05 - val_accuracy: 0.7929 - val_auc: 0.6622 - val_f1: 0.2811
Epoch 2/20
800/800 [==============================] - 23s 28ms/step - loss: 1.0815 - tp0.1: 2577878.0000 - fp0.1: 8005433.0000 - tn0.1: 8677535.0000 - fn0.1: 399954.0000 - precision0.1: 0.2436 - reca
ll0.1: 0.8657 - tp0.3: 1447126.0000 - fp0.3: 2241046.0000 - tn0.3: 14441922.0000 - fn0.3: 1530706.0000 - precision0.3: 0.3924 - recall0.3: 0.4860 - tp0.5: 687354.0000 - fp0.5: 398807.0000
- tn0.5: 16284161.0000 - fn0.5: 2290478.0000 - precision0.5: 0.6328 - recall0.5: 0.2308 - tp0.7: 413817.0000 - fp0.7: 98920.0000 - tn0.7: 16584048.0000 - fn0.7: 2564015.0000 - precision0.7
: 0.8071 - recall0.7: 0.1390 - tp0.9: 186771.0000 - fp0.9: 14141.0000 - tn0.9: 16668827.0000 - fn0.9: 2791061.0000 - precision0.9: 0.9296 - recall0.9: 0.0627 - accuracy: 0.8632 - auc: 0.77
95 - f1: 0.2631 - val_loss: 2.8986 - val_tp0.1: 803131.0000 - val_fp0.1: 4099762.0000 - val_tn0.1: 11581.0000 - val_fn0.1: 726.0000 - val_precision0.1: 0.1638 - val_recall0.1: 0.9991 - val
_tp0.3: 802096.0000 - val_fp0.3: 4084853.0000 - val_tn0.3: 26490.0000 - val_fn0.3: 1761.0000 - val_precision0.3: 0.1641 - val_recall0.3: 0.9978 - val_tp0.5: 651539.0000 - val_fp0.5: 192398
5.0000 - val_tn0.5: 2187358.0000 - val_fn0.5: 152318.0000 - val_precision0.5: 0.2530 - val_recall0.5: 0.8105 - val_tp0.7: 625381.0000 - val_fp0.7: 1805820.0000 - val_tn0.7: 2305523.0000 -
val_fn0.7: 178476.0000 - val_precision0.7: 0.2572 - val_recall0.7: 0.7780 - val_tp0.9: 585910.0000 - val_fp0.9: 1628567.0000 - val_tn0.9: 2482776.0000 - val_fn0.9: 217947.0000 - val_precis
ion0.9: 0.2646 - val_recall0.9: 0.7289 - val_accuracy: 0.5776 - val_auc: 0.7124 - val_f1: 0.2811
Epoch 3/20
800/800 [==============================] - 23s 28ms/step - loss: 1.0116 - tp0.1: 2624849.0000 - fp0.1: 7645828.0000 - tn0.1: 9037140.0000 - fn0.1: 352983.0000 - precision0.1: 0.2556 - reca
ll0.1: 0.8815 - tp0.3: 1498713.0000 - fp0.3: 1695358.0000 - tn0.3: 14987610.0000 - fn0.3: 1479119.0000 - precision0.3: 0.4692 - recall0.3: 0.5033 - tp0.5: 975584.0000 - fp0.5: 420254.0000
- tn0.5: 16262714.0000 - fn0.5: 2002248.0000 - precision0.5: 0.6989 - recall0.5: 0.3276 - tp0.7: 709358.0000 - fp0.7: 149271.0000 - tn0.7: 16533697.0000 - fn0.7: 2268474.0000 - precision0.
7: 0.8262 - recall0.7: 0.2382 - tp0.9: 389012.0000 - fp0.9: 27027.0000 - tn0.9: 16655941.0000 - fn0.9: 2588820.0000 - precision0.9: 0.9350 - recall0.9: 0.1306 - accuracy: 0.8768 - auc: 0.8
104 - f1: 0.2631 - val_loss: 1.5866 - val_tp0.1: 541567.0000 - val_fp0.1: 1371385.0000 - val_tn0.1: 2739958.0000 - val_fn0.1: 262290.0000 - val_precision0.1: 0.2831 - val_recall0.1: 0.6737
 - val_tp0.3: 350430.0000 - val_fp0.3: 695890.0000 - val_tn0.3: 3415453.0000 - val_fn0.3: 453427.0000 - val_precision0.3: 0.3349 - val_recall0.3: 0.4359 - val_tp0.5: 277275.0000 - val_fp0.
5: 441168.0000 - val_tn0.5: 3670175.0000 - val_fn0.5: 526582.0000 - val_precision0.5: 0.3859 - val_recall0.5: 0.3449 - val_tp0.7: 207293.0000 - val_fp0.7: 222426.0000 - val_tn0.7: 3888917.
0000 - val_fn0.7: 596564.0000 - val_precision0.7: 0.4824 - val_recall0.7: 0.2579 - val_tp0.9: 90589.0000 - val_fp0.9: 35047.0000 - val_tn0.9: 4076296.0000 - val_fn0.9: 713268.0000 - val_pr
ecision0.9: 0.7210 - val_recall0.9: 0.1127 - val_accuracy: 0.8031 - val_auc: 0.7058 - val_f1: 0.2811
Epoch 4/20
800/800 [==============================] - 23s 28ms/step - loss: 0.9534 - tp0.1: 2630735.0000 - fp0.1: 7252035.0000 - tn0.1: 9430933.0000 - fn0.1: 347097.0000 - precision0.1: 0.2662 - reca
ll0.1: 0.8834 - tp0.3: 1613584.0000 - fp0.3: 1430388.0000 - tn0.3: 15252580.0000 - fn0.3: 1364248.0000 - precision0.3: 0.5301 - recall0.3: 0.5419 - tp0.5: 1188917.0000 - fp0.5: 484302.0000
 - tn0.5: 16198666.0000 - fn0.5: 1788915.0000 - precision0.5: 0.7106 - recall0.5: 0.3993 - tp0.7: 896123.0000 - fp0.7: 181140.0000 - tn0.7: 16501828.0000 - fn0.7: 2081709.0000 - precision0
.7: 0.8319 - recall0.7: 0.3009 - tp0.9: 494120.0000 - fp0.9: 27734.0000 - tn0.9: 16655234.0000 - fn0.9: 2483712.0000 - precision0.9: 0.9469 - recall0.9: 0.1659 - accuracy: 0.8844 - auc: 0.
8330 - f1: 0.2631 - val_loss: 1.2238 - val_tp0.1: 762287.0000 - val_fp0.1: 2841918.0000 - val_tn0.1: 1269425.0000 - val_fn0.1: 41570.0000 - val_precision0.1: 0.2115 - val_recall0.1: 0.9483
 - val_tp0.3: 473341.0000 - val_fp0.3: 934680.0000 - val_tn0.3: 3176663.0000 - val_fn0.3: 330516.0000 - val_precision0.3: 0.3362 - val_recall0.3: 0.5888 - val_tp0.5: 284941.0000 - val_fp0.
5: 382747.0000 - val_tn0.5: 3728596.0000 - val_fn0.5: 518916.0000 - val_precision0.5: 0.4268 - val_recall0.5: 0.3545 - val_tp0.7: 199426.0000 - val_fp0.7: 215679.0000 - val_tn0.7: 3895664.
0000 - val_fn0.7: 604431.0000 - val_precision0.7: 0.4804 - val_recall0.7: 0.2481 - val_tp0.9: 80714.0000 - val_fp0.9: 40538.0000 - val_tn0.9: 4070805.0000 - val_fn0.9: 723143.0000 - val_pr
ecision0.9: 0.6657 - val_recall0.9: 0.1004 - val_accuracy: 0.8166 - val_auc: 0.7536 - val_f1: 0.2811
Epoch 5/20
800/800 [==============================] - 105s 132ms/step - loss: 0.9031 - tp0.1: 2615817.0000 - fp0.1: 6473866.0000 - tn0.1: 10209102.0000 - fn0.1: 362015.0000 - precision0.1: 0.2878 - r
ecall0.1: 0.8784 - tp0.3: 1760338.0000 - fp0.3: 1552636.0000 - tn0.3: 15130332.0000 - fn0.3: 1217494.0000 - precision0.3: 0.5313 - recall0.3: 0.5911 - tp0.5: 1305140.0000 - fp0.5: 538907.0
000 - tn0.5: 16144061.0000 - fn0.5: 1672692.0000 - precision0.5: 0.7078 - recall0.5: 0.4383 - tp0.7: 972649.0000 - fp0.7: 195218.0000 - tn0.7: 16487750.0000 - fn0.7: 2005183.0000 - precisi
on0.7: 0.8328 - recall0.7: 0.3266 - tp0.9: 541205.0000 - fp0.9: 31289.0000 - tn0.9: 16651679.0000 - fn0.9: 2436627.0000 - precision0.9: 0.9453 - recall0.9: 0.1817 - accuracy: 0.8875 - auc:
 0.8476 - f1: 0.2631 - val_loss: 1.0983 - val_tp0.1: 685124.0000 - val_fp0.1: 1862694.0000 - val_tn0.1: 2248649.0000 - val_fn0.1: 118733.0000 - val_precision0.1: 0.2689 - val_recall0.1: 0.
8523 - val_tp0.3: 408907.0000 - val_fp0.3: 510069.0000 - val_tn0.3: 3601274.0000 - val_fn0.3: 394950.0000 - val_precision0.3: 0.4450 - val_recall0.3: 0.5087 - val_tp0.5: 249171.0000 - val_
fp0.5: 222225.0000 - val_tn0.5: 3889118.0000 - val_fn0.5: 554686.0000 - val_precision0.5: 0.5286 - val_recall0.5: 0.3100 - val_tp0.7: 157112.0000 - val_fp0.7: 77659.0000 - val_tn0.7: 40336
84.0000 - val_fn0.7: 646745.0000 - val_precision0.7: 0.6692 - val_recall0.7: 0.1954 - val_tp0.9: 31306.0000 - val_fp0.9: 406.0000 - val_tn0.9: 4110937.0000 - val_fn0.9: 772551.0000 - val_p
recision0.9: 0.9872 - val_recall0.9: 0.0389 - val_accuracy: 0.8419 - val_auc: 0.7829 - val_f1: 0.2811
Epoch 6/20
800/800 [==============================] - 23s 28ms/step - loss: 0.8664 - tp0.1: 2613105.0000 - fp0.1: 5967389.0000 - tn0.1: 10715579.0000 - fn0.1: 364727.0000 - precision0.1: 0.3045 - rec
all0.1: 0.8775 - tp0.3: 1833388.0000 - fp0.3: 1537464.0000 - tn0.3: 15145504.0000 - fn0.3: 1144444.0000 - precision0.3: 0.5439 - recall0.3: 0.6157 - tp0.5: 1388900.0000 - fp0.5: 539537.000
0 - tn0.5: 16143431.0000 - fn0.5: 1588932.0000 - precision0.5: 0.7202 - recall0.5: 0.4664 - tp0.7: 1047260.0000 - fp0.7: 195978.0000 - tn0.7: 16486990.0000 - fn0.7: 1930572.0000 - precisio
n0.7: 0.8424 - recall0.7: 0.3517 - tp0.9: 612618.0000 - fp0.9: 34165.0000 - tn0.9: 16648803.0000 - fn0.9: 2365214.0000 - precision0.9: 0.9472 - recall0.9: 0.2057 - accuracy: 0.8917 - auc:
0.8558 - f1: 0.2631 - val_loss: 1.1906 - val_tp0.1: 589351.0000 - val_fp0.1: 1203883.0000 - val_tn0.1: 2907460.0000 - val_fn0.1: 214506.0000 - val_precision0.1: 0.3287 - val_recall0.1: 0.7
332 - val_tp0.3: 396327.0000 - val_fp0.3: 557062.0000 - val_tn0.3: 3554281.0000 - val_fn0.3: 407530.0000 - val_precision0.3: 0.4157 - val_recall0.3: 0.4930 - val_tp0.5: 335193.0000 - val_f
p0.5: 411860.0000 - val_tn0.5: 3699483.0000 - val_fn0.5: 468664.0000 - val_precision0.5: 0.4487 - val_recall0.5: 0.4170 - val_tp0.7: 273790.0000 - val_fp0.7: 273418.0000 - val_tn0.7: 38379
25.0000 - val_fn0.7: 530067.0000 - val_precision0.7: 0.5003 - val_recall0.7: 0.3406 - val_tp0.9: 147281.0000 - val_fp0.9: 47498.0000 - val_tn0.9: 4063845.0000 - val_fn0.9: 656576.0000 - va
l_precision0.9: 0.7561 - val_recall0.9: 0.1832 - val_accuracy: 0.8209 - val_auc: 0.7717 - val_f1: 0.2811
Epoch 7/20
800/800 [==============================] - 23s 29ms/step - loss: 0.8261 - tp0.1: 2610067.0000 - fp0.1: 5441450.0000 - tn0.1: 11241518.0000 - fn0.1: 367765.0000 - precision0.1: 0.3242 - rec
all0.1: 0.8765 - tp0.3: 1926083.0000 - fp0.3: 1638761.0000 - tn0.3: 15044207.0000 - fn0.3: 1051749.0000 - precision0.3: 0.5403 - recall0.3: 0.6468 - tp0.5: 1451422.0000 - fp0.5: 566878.000
0 - tn0.5: 16116090.0000 - fn0.5: 1526410.0000 - precision0.5: 0.7191 - recall0.5: 0.4874 - tp0.7: 1084565.0000 - fp0.7: 195437.0000 - tn0.7: 16487531.0000 - fn0.7: 1893267.0000 - precisio
n0.7: 0.8473 - recall0.7: 0.3642 - tp0.9: 629952.0000 - fp0.9: 32457.0000 - tn0.9: 16650511.0000 - fn0.9: 2347880.0000 - precision0.9: 0.9510 - recall0.9: 0.2115 - accuracy: 0.8935 - auc:
0.8646 - f1: 0.2631 - val_loss: 0.9781 - val_tp0.1: 706790.0000 - val_fp0.1: 1645956.0000 - val_tn0.1: 2465387.0000 - val_fn0.1: 97067.0000 - val_precision0.1: 0.3004 - val_recall0.1: 0.87
92 - val_tp0.3: 517245.0000 - val_fp0.3: 647485.0000 - val_tn0.3: 3463858.0000 - val_fn0.3: 286612.0000 - val_precision0.3: 0.4441 - val_recall0.3: 0.6435 - val_tp0.5: 373721.0000 - val_fp
0.5: 255513.0000 - val_tn0.5: 3855830.0000 - val_fn0.5: 430136.0000 - val_precision0.5: 0.5939 - val_recall0.5: 0.4649 - val_tp0.7: 242514.0000 - val_fp0.7: 84275.0000 - val_tn0.7: 4027068
.0000 - val_fn0.7: 561343.0000 - val_precision0.7: 0.7421 - val_recall0.7: 0.3017 - val_tp0.9: 127788.0000 - val_fp0.9: 15373.0000 - val_tn0.9: 4095970.0000 - val_fn0.9: 676069.0000 - val_
precision0.9: 0.8926 - val_recall0.9: 0.1590 - val_accuracy: 0.8605 - val_auc: 0.8289 - val_f1: 0.2811
Epoch 8/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7968 - tp0.1: 2628693.0000 - fp0.1: 5237949.0000 - tn0.1: 11445019.0000 - fn0.1: 349139.0000 - precision0.1: 0.3342 - rec
all0.1: 0.8828 - tp0.3: 1990774.0000 - fp0.3: 1601561.0000 - tn0.3: 15081407.0000 - fn0.3: 987058.0000 - precision0.3: 0.5542 - recall0.3: 0.6685 - tp0.5: 1512649.0000 - fp0.5: 585437.0000
 - tn0.5: 16097531.0000 - fn0.5: 1465183.0000 - precision0.5: 0.7210 - recall0.5: 0.5080 - tp0.7: 1113359.0000 - fp0.7: 192219.0000 - tn0.7: 16490749.0000 - fn0.7: 1864473.0000 - precision
0.7: 0.8528 - recall0.7: 0.3739 - tp0.9: 662809.0000 - fp0.9: 30877.0000 - tn0.9: 16652091.0000 - fn0.9: 2315023.0000 - precision0.9: 0.9555 - recall0.9: 0.2226 - accuracy: 0.8957 - auc: 0
.8732 - f1: 0.2631 - val_loss: 1.0262 - val_tp0.1: 703581.0000 - val_fp0.1: 1905662.0000 - val_tn0.1: 2205681.0000 - val_fn0.1: 100276.0000 - val_precision0.1: 0.2696 - val_recall0.1: 0.87
53 - val_tp0.3: 447413.0000 - val_fp0.3: 447251.0000 - val_tn0.3: 3664092.0000 - val_fn0.3: 356444.0000 - val_precision0.3: 0.5001 - val_recall0.3: 0.5566 - val_tp0.5: 293285.0000 - val_fp
0.5: 134465.0000 - val_tn0.5: 3976878.0000 - val_fn0.5: 510572.0000 - val_precision0.5: 0.6856 - val_recall0.5: 0.3648 - val_tp0.7: 180739.0000 - val_fp0.7: 40494.0000 - val_tn0.7: 4070849
.0000 - val_fn0.7: 623118.0000 - val_precision0.7: 0.8170 - val_recall0.7: 0.2248 - val_tp0.9: 86388.0000 - val_fp0.9: 5793.0000 - val_tn0.9: 4105550.0000 - val_fn0.9: 717469.0000 - val_pr
ecision0.9: 0.9372 - val_recall0.9: 0.1075 - val_accuracy: 0.8688 - val_auc: 0.8159 - val_f1: 0.2811
Epoch 9/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7819 - tp0.1: 2640898.0000 - fp0.1: 5239761.0000 - tn0.1: 11443207.0000 - fn0.1: 336934.0000 - precision0.1: 0.3351 - rec
all0.1: 0.8869 - tp0.3: 2019563.0000 - fp0.3: 1566552.0000 - tn0.3: 15116416.0000 - fn0.3: 958269.0000 - precision0.3: 0.5632 - recall0.3: 0.6782 - tp0.5: 1571033.0000 - fp0.5: 592615.0000
 - tn0.5: 16090353.0000 - fn0.5: 1406799.0000 - precision0.5: 0.7261 - recall0.5: 0.5276 - tp0.7: 1175144.0000 - fp0.7: 194680.0000 - tn0.7: 16488288.0000 - fn0.7: 1802688.0000 - precision
0.7: 0.8579 - recall0.7: 0.3946 - tp0.9: 742669.0000 - fp0.9: 35986.0000 - tn0.9: 16646982.0000 - fn0.9: 2235163.0000 - precision0.9: 0.9538 - recall0.9: 0.2494 - accuracy: 0.8983 - auc: 0
.8780 - f1: 0.2631 - val_loss: 1.0157 - val_tp0.1: 652637.0000 - val_fp0.1: 1329883.0000 - val_tn0.1: 2781460.0000 - val_fn0.1: 151220.0000 - val_precision0.1: 0.3292 - val_recall0.1: 0.81
19 - val_tp0.3: 420948.0000 - val_fp0.3: 353053.0000 - val_tn0.3: 3758290.0000 - val_fn0.3: 382909.0000 - val_precision0.3: 0.5439 - val_recall0.3: 0.5237 - val_tp0.5: 281081.0000 - val_fp
0.5: 123121.0000 - val_tn0.5: 3988222.0000 - val_fn0.5: 522776.0000 - val_precision0.5: 0.6954 - val_recall0.5: 0.3497 - val_tp0.7: 173940.0000 - val_fp0.7: 36200.0000 - val_tn0.7: 4075143
.0000 - val_fn0.7: 629917.0000 - val_precision0.7: 0.8277 - val_recall0.7: 0.2164 - val_tp0.9: 89015.0000 - val_fp0.9: 6773.0000 - val_tn0.9: 4104570.0000 - val_fn0.9: 714842.0000 - val_pr
ecision0.9: 0.9293 - val_recall0.9: 0.1107 - val_accuracy: 0.8686 - val_auc: 0.8166 - val_f1: 0.2811
Epoch 10/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7672 - tp0.1: 2635701.0000 - fp0.1: 4919057.0000 - tn0.1: 11763911.0000 - fn0.1: 342131.0000 - precision0.1: 0.3489 - rec
all0.1: 0.8851 - tp0.3: 2036938.0000 - fp0.3: 1575677.0000 - tn0.3: 15107291.0000 - fn0.3: 940894.0000 - precision0.3: 0.5638 - recall0.3: 0.6840 - tp0.5: 1588676.0000 - fp0.5: 624407.0000
 - tn0.5: 16058561.0000 - fn0.5: 1389156.0000 - precision0.5: 0.7179 - recall0.5: 0.5335 - tp0.7: 1150076.0000 - fp0.7: 189388.0000 - tn0.7: 16493580.0000 - fn0.7: 1827756.0000 - precision
0.7: 0.8586 - recall0.7: 0.3862 - tp0.9: 694637.0000 - fp0.9: 31558.0000 - tn0.9: 16651410.0000 - fn0.9: 2283195.0000 - precision0.9: 0.9565 - recall0.9: 0.2333 - accuracy: 0.8976 - auc: 0
.8799 - f1: 0.2631 - val_loss: 0.9685 - val_tp0.1: 719258.0000 - val_fp0.1: 1852284.0000 - val_tn0.1: 2259059.0000 - val_fn0.1: 84599.0000 - val_precision0.1: 0.2797 - val_recall0.1: 0.894
8 - val_tp0.3: 495938.0000 - val_fp0.3: 497006.0000 - val_tn0.3: 3614337.0000 - val_fn0.3: 307919.0000 - val_precision0.3: 0.4995 - val_recall0.3: 0.6169 - val_tp0.5: 339199.0000 - val_fp0
.5: 177289.0000 - val_tn0.5: 3934054.0000 - val_fn0.5: 464658.0000 - val_precision0.5: 0.6567 - val_recall0.5: 0.4220 - val_tp0.7: 196231.0000 - val_fp0.7: 44292.0000 - val_tn0.7: 4067051.
0000 - val_fn0.7: 607626.0000 - val_precision0.7: 0.8159 - val_recall0.7: 0.2441 - val_tp0.9: 84223.0000 - val_fp0.9: 4726.0000 - val_tn0.9: 4106617.0000 - val_fn0.9: 719634.0000 - val_pre
cision0.9: 0.9469 - val_recall0.9: 0.1048 - val_accuracy: 0.8694 - val_auc: 0.8365 - val_f1: 0.2811
Epoch 11/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7657 - tp0.1: 2620304.0000 - fp0.1: 4756693.0000 - tn0.1: 11926275.0000 - fn0.1: 357528.0000 - precision0.1: 0.3552 - rec
all0.1: 0.8799 - tp0.3: 2057463.0000 - fp0.3: 1595402.0000 - tn0.3: 15087566.0000 - fn0.3: 920369.0000 - precision0.3: 0.5632 - recall0.3: 0.6909 - tp0.5: 1629457.0000 - fp0.5: 652845.0000
 - tn0.5: 16030123.0000 - fn0.5: 1348375.0000 - precision0.5: 0.7140 - recall0.5: 0.5472 - tp0.7: 1159302.0000 - fp0.7: 181594.0000 - tn0.7: 16501374.0000 - fn0.7: 1818530.0000 - precision
0.7: 0.8646 - recall0.7: 0.3893 - tp0.9: 691411.0000 - fp0.9: 29139.0000 - tn0.9: 16653829.0000 - fn0.9: 2286421.0000 - precision0.9: 0.9596 - recall0.9: 0.2322 - accuracy: 0.8982 - auc: 0
.8794 - f1: 0.2631 - val_loss: 0.9798 - val_tp0.1: 650390.0000 - val_fp0.1: 1145138.0000 - val_tn0.1: 2966205.0000 - val_fn0.1: 153467.0000 - val_precision0.1: 0.3622 - val_recall0.1: 0.80
91 - val_tp0.3: 416815.0000 - val_fp0.3: 345286.0000 - val_tn0.3: 3766057.0000 - val_fn0.3: 387042.0000 - val_precision0.3: 0.5469 - val_recall0.3: 0.5185 - val_tp0.5: 303813.0000 - val_fp
0.5: 119881.0000 - val_tn0.5: 3991462.0000 - val_fn0.5: 500044.0000 - val_precision0.5: 0.7171 - val_recall0.5: 0.3779 - val_tp0.7: 186405.0000 - val_fp0.7: 34348.0000 - val_tn0.7: 4076995
.0000 - val_fn0.7: 617452.0000 - val_precision0.7: 0.8444 - val_recall0.7: 0.2319 - val_tp0.9: 88035.0000 - val_fp0.9: 4840.0000 - val_tn0.9: 4106503.0000 - val_fn0.9: 715822.0000 - val_pr
ecision0.9: 0.9479 - val_recall0.9: 0.1095 - val_accuracy: 0.8739 - val_auc: 0.8236 - val_f1: 0.2811
Epoch 12/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7483 - tp0.1: 2643606.0000 - fp0.1: 4772914.0000 - tn0.1: 11910054.0000 - fn0.1: 334226.0000 - precision0.1: 0.3564 - rec
all0.1: 0.8878 - tp0.3: 2076874.0000 - fp0.3: 1529573.0000 - tn0.3: 15153395.0000 - fn0.3: 900958.0000 - precision0.3: 0.5759 - recall0.3: 0.6974 - tp0.5: 1673377.0000 - fp0.5: 652544.0000
 - tn0.5: 16030424.0000 - fn0.5: 1304455.0000 - precision0.5: 0.7194 - recall0.5: 0.5619 - tp0.7: 1197669.0000 - fp0.7: 195301.0000 - tn0.7: 16487667.0000 - fn0.7: 1780163.0000 - precision
0.7: 0.8598 - recall0.7: 0.4022 - tp0.9: 723663.0000 - fp0.9: 34095.0000 - tn0.9: 16648873.0000 - fn0.9: 2254169.0000 - precision0.9: 0.9550 - recall0.9: 0.2430 - accuracy: 0.9005 - auc: 0
.8854 - f1: 0.2631 - val_loss: 0.9661 - val_tp0.1: 682417.0000 - val_fp0.1: 1411604.0000 - val_tn0.1: 2699739.0000 - val_fn0.1: 121440.0000 - val_precision0.1: 0.3259 - val_recall0.1: 0.84
89 - val_tp0.3: 393462.0000 - val_fp0.3: 248966.0000 - val_tn0.3: 3862377.0000 - val_fn0.3: 410395.0000 - val_precision0.3: 0.6125 - val_recall0.3: 0.4895 - val_tp0.5: 264605.0000 - val_fp
0.5: 75544.0000 - val_tn0.5: 4035799.0000 - val_fn0.5: 539252.0000 - val_precision0.5: 0.7779 - val_recall0.5: 0.3292 - val_tp0.7: 151048.0000 - val_fp0.7: 18110.0000 - val_tn0.7: 4093233.
0000 - val_fn0.7: 652809.0000 - val_precision0.7: 0.8929 - val_recall0.7: 0.1879 - val_tp0.9: 73096.0000 - val_fp0.9: 2503.0000 - val_tn0.9: 4108840.0000 - val_fn0.9: 730761.0000 - val_pre
cision0.9: 0.9669 - val_recall0.9: 0.0909 - val_accuracy: 0.8749 - val_auc: 0.8348 - val_f1: 0.2811
Epoch 13/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7469 - tp0.1: 2638319.0000 - fp0.1: 4684335.0000 - tn0.1: 11998633.0000 - fn0.1: 339513.0000 - precision0.1: 0.3603 - rec
all0.1: 0.8860 - tp0.3: 2075631.0000 - fp0.3: 1533614.0000 - tn0.3: 15149354.0000 - fn0.3: 902201.0000 - precision0.3: 0.5751 - recall0.3: 0.6970 - tp0.5: 1654093.0000 - fp0.5: 657052.0000
 - tn0.5: 16025916.0000 - fn0.5: 1323739.0000 - precision0.5: 0.7157 - recall0.5: 0.5555 - tp0.7: 1159144.0000 - fp0.7: 187558.0000 - tn0.7: 16495410.0000 - fn0.7: 1818688.0000 - precision
0.7: 0.8607 - recall0.7: 0.3893 - tp0.9: 691098.0000 - fp0.9: 30329.0000 - tn0.9: 16652639.0000 - fn0.9: 2286734.0000 - precision0.9: 0.9580 - recall0.9: 0.2321 - accuracy: 0.8993 - auc: 0
.8851 - f1: 0.2631 - val_loss: 0.9511 - val_tp0.1: 676012.0000 - val_fp0.1: 1300131.0000 - val_tn0.1: 2811212.0000 - val_fn0.1: 127845.0000 - val_precision0.1: 0.3421 - val_recall0.1: 0.84
10 - val_tp0.3: 435613.0000 - val_fp0.3: 369776.0000 - val_tn0.3: 3741567.0000 - val_fn0.3: 368244.0000 - val_precision0.3: 0.5409 - val_recall0.3: 0.5419 - val_tp0.5: 315382.0000 - val_fp
0.5: 126324.0000 - val_tn0.5: 3985019.0000 - val_fn0.5: 488475.0000 - val_precision0.5: 0.7140 - val_recall0.5: 0.3923 - val_tp0.7: 186834.0000 - val_fp0.7: 33643.0000 - val_tn0.7: 4077700
.0000 - val_fn0.7: 617023.0000 - val_precision0.7: 0.8474 - val_recall0.7: 0.2324 - val_tp0.9: 86226.0000 - val_fp0.9: 4347.0000 - val_tn0.9: 4106996.0000 - val_fn0.9: 717631.0000 - val_pr
ecision0.9: 0.9520 - val_recall0.9: 0.1073 - val_accuracy: 0.8749 - val_auc: 0.8317 - val_f1: 0.2811
Epoch 14/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7472 - tp0.1: 2640336.0000 - fp0.1: 4682320.0000 - tn0.1: 12000648.0000 - fn0.1: 337496.0000 - precision0.1: 0.3606 - rec
all0.1: 0.8867 - tp0.3: 2072009.0000 - fp0.3: 1595973.0000 - tn0.3: 15086995.0000 - fn0.3: 905823.0000 - precision0.3: 0.5649 - recall0.3: 0.6958 - tp0.5: 1664556.0000 - fp0.5: 686406.0000
 - tn0.5: 15996562.0000 - fn0.5: 1313276.0000 - precision0.5: 0.7080 - recall0.5: 0.5590 - tp0.7: 1168681.0000 - fp0.7: 190677.0000 - tn0.7: 16492291.0000 - fn0.7: 1809151.0000 - precision
0.7: 0.8597 - recall0.7: 0.3925 - tp0.9: 706963.0000 - fp0.9: 30254.0000 - tn0.9: 16652714.0000 - fn0.9: 2270869.0000 - precision0.9: 0.9590 - recall0.9: 0.2374 - accuracy: 0.8983 - auc: 0
.8838 - f1: 0.2631 - val_loss: 0.9537 - val_tp0.1: 687713.0000 - val_fp0.1: 1442587.0000 - val_tn0.1: 2668756.0000 - val_fn0.1: 116144.0000 - val_precision0.1: 0.3228 - val_recall0.1: 0.85
55 - val_tp0.3: 448956.0000 - val_fp0.3: 376999.0000 - val_tn0.3: 3734344.0000 - val_fn0.3: 354901.0000 - val_precision0.3: 0.5436 - val_recall0.3: 0.5585 - val_tp0.5: 322122.0000 - val_fp
0.5: 133228.0000 - val_tn0.5: 3978115.0000 - val_fn0.5: 481735.0000 - val_precision0.5: 0.7074 - val_recall0.5: 0.4007 - val_tp0.7: 189079.0000 - val_fp0.7: 36378.0000 - val_tn0.7: 4074965
.0000 - val_fn0.7: 614778.0000 - val_precision0.7: 0.8386 - val_recall0.7: 0.2352 - val_tp0.9: 87530.0000 - val_fp0.9: 4845.0000 - val_tn0.9: 4106498.0000 - val_fn0.9: 716327.0000 - val_pr
ecision0.9: 0.9476 - val_recall0.9: 0.1089 - val_accuracy: 0.8749 - val_auc: 0.8339 - val_f1: 0.2811
Epoch 15/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7436 - tp0.1: 2645410.0000 - fp0.1: 4733153.0000 - tn0.1: 11949815.0000 - fn0.1: 332422.0000 - precision0.1: 0.3585 - rec
all0.1: 0.8884 - tp0.3: 2099091.0000 - fp0.3: 1622697.0000 - tn0.3: 15060271.0000 - fn0.3: 878741.0000 - precision0.3: 0.5640 - recall0.3: 0.7049 - tp0.5: 1689821.0000 - fp0.5: 706002.0000
 - tn0.5: 15976966.0000 - fn0.5: 1288011.0000 - precision0.5: 0.7053 - recall0.5: 0.5675 - tp0.7: 1187271.0000 - fp0.7: 194662.0000 - tn0.7: 16488306.0000 - fn0.7: 1790561.0000 - precision
0.7: 0.8591 - recall0.7: 0.3987 - tp0.9: 719432.0000 - fp0.9: 31350.0000 - tn0.9: 16651618.0000 - fn0.9: 2258400.0000 - precision0.9: 0.9582 - recall0.9: 0.2416 - accuracy: 0.8986 - auc: 0
.8855 - f1: 0.2631 - val_loss: 0.9517 - val_tp0.1: 686003.0000 - val_fp0.1: 1413980.0000 - val_tn0.1: 2697363.0000 - val_fn0.1: 117854.0000 - val_precision0.1: 0.3267 - val_recall0.1: 0.85
34 - val_tp0.3: 449760.0000 - val_fp0.3: 382567.0000 - val_tn0.3: 3728776.0000 - val_fn0.3: 354097.0000 - val_precision0.3: 0.5404 - val_recall0.3: 0.5595 - val_tp0.5: 324012.0000 - val_fp
0.5: 130173.0000 - val_tn0.5: 3981170.0000 - val_fn0.5: 479845.0000 - val_precision0.5: 0.7134 - val_recall0.5: 0.4031 - val_tp0.7: 189632.0000 - val_fp0.7: 34603.0000 - val_tn0.7: 4076740
.0000 - val_fn0.7: 614225.0000 - val_precision0.7: 0.8457 - val_recall0.7: 0.2359 - val_tp0.9: 86754.0000 - val_fp0.9: 4383.0000 - val_tn0.9: 4106960.0000 - val_fn0.9: 717103.0000 - val_pr
ecision0.9: 0.9519 - val_recall0.9: 0.1079 - val_accuracy: 0.8759 - val_auc: 0.8345 - val_f1: 0.2811
Epoch 16/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7458 - tp0.1: 2641645.0000 - fp0.1: 4695396.0000 - tn0.1: 11987572.0000 - fn0.1: 336187.0000 - precision0.1: 0.3600 - rec
all0.1: 0.8871 - tp0.3: 2084584.0000 - fp0.3: 1588554.0000 - tn0.3: 15094414.0000 - fn0.3: 893248.0000 - precision0.3: 0.5675 - recall0.3: 0.7000 - tp0.5: 1683391.0000 - fp0.5: 686943.0000
 - tn0.5: 15996025.0000 - fn0.5: 1294441.0000 - precision0.5: 0.7102 - recall0.5: 0.5653 - tp0.7: 1187166.0000 - fp0.7: 191714.0000 - tn0.7: 16491254.0000 - fn0.7: 1790666.0000 - precision
0.7: 0.8610 - recall0.7: 0.3987 - tp0.9: 713157.0000 - fp0.9: 30164.0000 - tn0.9: 16652804.0000 - fn0.9: 2264675.0000 - precision0.9: 0.9594 - recall0.9: 0.2395 - accuracy: 0.8992 - auc: 0
.8851 - f1: 0.2631 - val_loss: 0.9480 - val_tp0.1: 687202.0000 - val_fp0.1: 1416765.0000 - val_tn0.1: 2694578.0000 - val_fn0.1: 116655.0000 - val_precision0.1: 0.3266 - val_recall0.1: 0.85
49 - val_tp0.3: 435185.0000 - val_fp0.3: 324375.0000 - val_tn0.3: 3786968.0000 - val_fn0.3: 368672.0000 - val_precision0.3: 0.5729 - val_recall0.3: 0.5414 - val_tp0.5: 310697.0000 - val_fp
0.5: 114008.0000 - val_tn0.5: 3997335.0000 - val_fn0.5: 493160.0000 - val_precision0.5: 0.7316 - val_recall0.5: 0.3865 - val_tp0.7: 180896.0000 - val_fp0.7: 30514.0000 - val_tn0.7: 4080829
.0000 - val_fn0.7: 622961.0000 - val_precision0.7: 0.8557 - val_recall0.7: 0.2250 - val_tp0.9: 85121.0000 - val_fp0.9: 4248.0000 - val_tn0.9: 4107095.0000 - val_fn0.9: 718736.0000 - val_pr
ecision0.9: 0.9525 - val_recall0.9: 0.1059 - val_accuracy: 0.8765 - val_auc: 0.8372 - val_f1: 0.2811
Epoch 17/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7489 - tp0.1: 2637863.0000 - fp0.1: 4686124.0000 - tn0.1: 11996844.0000 - fn0.1: 339969.0000 - precision0.1: 0.3602 - rec
all0.1: 0.8858 - tp0.3: 2075559.0000 - fp0.3: 1564588.0000 - tn0.3: 15118380.0000 - fn0.3: 902273.0000 - precision0.3: 0.5702 - recall0.3: 0.6970 - tp0.5: 1674724.0000 - fp0.5: 689631.0000
 - tn0.5: 15993337.0000 - fn0.5: 1303108.0000 - precision0.5: 0.7083 - recall0.5: 0.5624 - tp0.7: 1179885.0000 - fp0.7: 191863.0000 - tn0.7: 16491105.0000 - fn0.7: 1797947.0000 - precision
0.7: 0.8601 - recall0.7: 0.3962 - tp0.9: 710902.0000 - fp0.9: 31154.0000 - tn0.9: 16651814.0000 - fn0.9: 2266930.0000 - precision0.9: 0.9580 - recall0.9: 0.2387 - accuracy: 0.8986 - auc: 0
.8842 - f1: 0.2631 - val_loss: 0.9526 - val_tp0.1: 685383.0000 - val_fp0.1: 1407315.0000 - val_tn0.1: 2704028.0000 - val_fn0.1: 118474.0000 - val_precision0.1: 0.3275 - val_recall0.1: 0.85
26 - val_tp0.3: 434253.0000 - val_fp0.3: 334816.0000 - val_tn0.3: 3776527.0000 - val_fn0.3: 369604.0000 - val_precision0.3: 0.5646 - val_recall0.3: 0.5402 - val_tp0.5: 309436.0000 - val_fp
0.5: 113860.0000 - val_tn0.5: 3997483.0000 - val_fn0.5: 494421.0000 - val_precision0.5: 0.7310 - val_recall0.5: 0.3849 - val_tp0.7: 180756.0000 - val_fp0.7: 30132.0000 - val_tn0.7: 4081211
.0000 - val_fn0.7: 623101.0000 - val_precision0.7: 0.8571 - val_recall0.7: 0.2249 - val_tp0.9: 85627.0000 - val_fp0.9: 4210.0000 - val_tn0.9: 4107133.0000 - val_fn0.9: 718230.0000 - val_pr
ecision0.9: 0.9531 - val_recall0.9: 0.1065 - val_accuracy: 0.8762 - val_auc: 0.8351 - val_f1: 0.2811
Epoch 18/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7393 - tp0.1: 2648936.0000 - fp0.1: 4669640.0000 - tn0.1: 12013328.0000 - fn0.1: 328896.0000 - precision0.1: 0.3619 - rec
all0.1: 0.8896 - tp0.3: 2088022.0000 - fp0.3: 1535327.0000 - tn0.3: 15147641.0000 - fn0.3: 889810.0000 - precision0.3: 0.5763 - recall0.3: 0.7012 - tp0.5: 1675010.0000 - fp0.5: 662872.0000
 - tn0.5: 16020096.0000 - fn0.5: 1302822.0000 - precision0.5: 0.7165 - recall0.5: 0.5625 - tp0.7: 1182736.0000 - fp0.7: 184372.0000 - tn0.7: 16498596.0000 - fn0.7: 1795096.0000 - precision
0.7: 0.8651 - recall0.7: 0.3972 - tp0.9: 719695.0000 - fp0.9: 31350.0000 - tn0.9: 16651618.0000 - fn0.9: 2258137.0000 - precision0.9: 0.9583 - recall0.9: 0.2417 - accuracy: 0.9000 - auc: 0
.8873 - f1: 0.2631 - val_loss: 0.9553 - val_tp0.1: 682153.0000 - val_fp0.1: 1382728.0000 - val_tn0.1: 2728615.0000 - val_fn0.1: 121704.0000 - val_precision0.1: 0.3304 - val_recall0.1: 0.84
86 - val_tp0.3: 433097.0000 - val_fp0.3: 334545.0000 - val_tn0.3: 3776798.0000 - val_fn0.3: 370760.0000 - val_precision0.3: 0.5642 - val_recall0.3: 0.5388 - val_tp0.5: 307911.0000 - val_fp
0.5: 113008.0000 - val_tn0.5: 3998335.0000 - val_fn0.5: 495946.0000 - val_precision0.5: 0.7315 - val_recall0.5: 0.3830 - val_tp0.7: 178664.0000 - val_fp0.7: 29733.0000 - val_tn0.7: 4081610
.0000 - val_fn0.7: 625193.0000 - val_precision0.7: 0.8573 - val_recall0.7: 0.2223 - val_tp0.9: 83772.0000 - val_fp0.9: 4111.0000 - val_tn0.9: 4107232.0000 - val_fn0.9: 720085.0000 - val_pr
ecision0.9: 0.9532 - val_recall0.9: 0.1042 - val_accuracy: 0.8761 - val_auc: 0.8340 - val_f1: 0.2811
Epoch 19/20
800/800 [==============================] - 23s 28ms/step - loss: 0.7452 - tp0.1: 2646665.0000 - fp0.1: 4698587.0000 - tn0.1: 11984381.0000 - fn0.1: 331167.0000 - precision0.1: 0.3603 - rec
all0.1: 0.8888 - tp0.3: 2062641.0000 - fp0.3: 1557707.0000 - tn0.3: 15125261.0000 - fn0.3: 915191.0000 - precision0.3: 0.5697 - recall0.3: 0.6927 - tp0.5: 1648838.0000 - fp0.5: 680800.0000
 - tn0.5: 16002168.0000 - fn0.5: 1328994.0000 - precision0.5: 0.7078 - recall0.5: 0.5537 - tp0.7: 1164585.0000 - fp0.7: 186942.0000 - tn0.7: 16496026.0000 - fn0.7: 1813247.0000 - precision
0.7: 0.8617 - recall0.7: 0.3911 - tp0.9: 705028.0000 - fp0.9: 30241.0000 - tn0.9: 16652727.0000 - fn0.9: 2272804.0000 - precision0.9: 0.9589 - recall0.9: 0.2368 - accuracy: 0.8978 - auc: 0
.8850 - f1: 0.2631 - val_loss: 0.9544 - val_tp0.1: 689367.0000 - val_fp0.1: 1469991.0000 - val_tn0.1: 2641352.0000 - val_fn0.1: 114490.0000 - val_precision0.1: 0.3192 - val_recall0.1: 0.85
76 - val_tp0.3: 451534.0000 - val_fp0.3: 363328.0000 - val_tn0.3: 3748015.0000 - val_fn0.3: 352323.0000 - val_precision0.3: 0.5541 - val_recall0.3: 0.5617 - val_tp0.5: 324284.0000 - val_fp
0.5: 128372.0000 - val_tn0.5: 3982971.0000 - val_fn0.5: 479573.0000 - val_precision0.5: 0.7164 - val_recall0.5: 0.4034 - val_tp0.7: 190210.0000 - val_fp0.7: 35390.0000 - val_tn0.7: 4075953
.0000 - val_fn0.7: 613647.0000 - val_precision0.7: 0.8431 - val_recall0.7: 0.2366 - val_tp0.9: 88933.0000 - val_fp0.9: 4874.0000 - val_tn0.9: 4106469.0000 - val_fn0.9: 714924.0000 - val_pr
ecision0.9: 0.9480 - val_recall0.9: 0.1106 - val_accuracy: 0.8763 - val_auc: 0.8353 - val_f1: 0.2811
Epoch 20/20
800/800 [==============================] - 22s 28ms/step - loss: 0.7364 - tp0.1: 2654787.0000 - fp0.1: 4675725.0000 - tn0.1: 12007243.0000 - fn0.1: 323045.0000 - precision0.1: 0.3622 - rec
all0.1: 0.8915 - tp0.3: 2081760.0000 - fp0.3: 1548676.0000 - tn0.3: 15134292.0000 - fn0.3: 896072.0000 - precision0.3: 0.5734 - recall0.3: 0.6991 - tp0.5: 1666321.0000 - fp0.5: 674703.0000
 - tn0.5: 16008265.0000 - fn0.5: 1311511.0000 - precision0.5: 0.7118 - recall0.5: 0.5596 - tp0.7: 1164069.0000 - fp0.7: 189776.0000 - tn0.7: 16493192.0000 - fn0.7: 1813763.0000 - precision
0.7: 0.8598 - recall0.7: 0.3909 - tp0.9: 699952.0000 - fp0.9: 31138.0000 - tn0.9: 16651830.0000 - fn0.9: 2277880.0000 - precision0.9: 0.9574 - recall0.9: 0.2351 - accuracy: 0.8990 - auc: 0
.8875 - f1: 0.2631 - val_loss: 0.9490 - val_tp0.1: 690953.0000 - val_fp0.1: 1466947.0000 - val_tn0.1: 2644396.0000 - val_fn0.1: 112904.0000 - val_precision0.1: 0.3202 - val_recall0.1: 0.85
95 - val_tp0.3: 446418.0000 - val_fp0.3: 345263.0000 - val_tn0.3: 3766080.0000 - val_fn0.3: 357439.0000 - val_precision0.3: 0.5639 - val_recall0.3: 0.5553 - val_tp0.5: 321059.0000 - val_fp
0.5: 123555.0000 - val_tn0.5: 3987788.0000 - val_fn0.5: 482798.0000 - val_precision0.5: 0.7221 - val_recall0.5: 0.3994 - val_tp0.7: 189576.0000 - val_fp0.7: 35232.0000 - val_tn0.7: 4076111
.0000 - val_fn0.7: 614281.0000 - val_precision0.7: 0.8433 - val_recall0.7: 0.2358 - val_tp0.9: 90962.0000 - val_fp0.9: 5426.0000 - val_tn0.9: 4105917.0000 - val_fn0.9: 712895.0000 - val_pr
ecision0.9: 0.9437 - val_recall0.9: 0.1132 - val_accuracy: 0.8766 - val_auc: 0.8372 - val_f1: 0.2811
--- Running training session 61/140
{'hp_epochs': 20, 'hp_batch_size': 20, 'hp_scaler': 'standard', 'hp_n_levels': 3, 'hp_first_filters': 128, 'hp_pool_size': 4, 'hp_input_size': 16384, 'hp_lr_start': 0.04354970735327304, 'h
p_lr_power': 1.0}
--- repeat #: 1
input - shape:   (None, 16384, 1)
output - shape:  (None, 16384, 1)
Epoch 1/20
240/240 [==============================] - 145s 530ms/step - loss: 0.7930 - tp0.1: 9902012.0000 - fp0.1: 12359941.0000 - tn0.1: 54313476.0000 - fn0.1: 2067755.0000 - precision0.1: 0.4448 -
 recall0.1: 0.8273 - tp0.3: 8813463.0000 - fp0.3: 7147978.0000 - tn0.3: 59525464.0000 - fn0.3: 3156304.0000 - precision0.3: 0.5522 - recall0.3: 0.7363 - tp0.5: 6775614.0000 - fp0.5: 236477
6.0000 - tn0.5: 64308628.0000 - fn0.5: 5194153.0000 - precision0.5: 0.7413 - recall0.5: 0.5661 - tp0.7: 5269669.0000 - fp0.7: 804460.0000 - tn0.7: 65868980.0000 - fn0.7: 6700098.0000 - pre
cision0.7: 0.8676 - recall0.7: 0.4402 - tp0.9: 3642391.0000 - fp0.9: 181910.0000 - tn0.9: 66491528.0000 - fn0.9: 8327376.0000 - precision0.9: 0.9524 - recall0.9: 0.3043 - accuracy: 0.9039
- auc: 0.8736 - f1: 0.2642 - val_loss: 1.3831 - val_tp0.1: 1860092.0000 - val_fp0.1: 529606.0000 - val_tn0.1: 15968340.0000 - val_fn0.1: 1302762.0000 - val_precision0.1: 0.7784 - val_recal
l0.1: 0.5881 - val_tp0.3: 1805849.0000 - val_fp0.3: 437765.0000 - val_tn0.3: 16060181.0000 - val_fn0.3: 1357005.0000 - val_precision0.3: 0.8049 - val_recall0.3: 0.5710 - val_tp0.5: 1737224
.0000 - val_fp0.5: 360788.0000 - val_tn0.5: 16137158.0000 - val_fn0.5: 1425630.0000 - val_precision0.5: 0.8280 - val_recall0.5: 0.5493 - val_tp0.7: 1675344.0000 - val_fp0.7: 306812.0000 -
val_tn0.7: 16191134.0000 - val_fn0.7: 1487510.0000 - val_precision0.7: 0.8452 - val_recall0.7: 0.5297 - val_tp0.9: 1555952.0000 - val_fp0.9: 227670.0000 - val_tn0.9: 16270276.0000 - val_fn
0.9: 1606902.0000 - val_precision0.9: 0.8724 - val_recall0.9: 0.4919 - val_accuracy: 0.9091 - val_auc: 0.7866 - val_f1: 0.2772
Epoch 2/20
240/240 [==============================] - 123s 514ms/step - loss: 0.6145 - tp0.1: 10414504.0000 - fp0.1: 10347179.0000 - tn0.1: 56326272.0000 - fn0.1: 1555263.0000 - precision0.1: 0.5016
- recall0.1: 0.8701 - tp0.3: 9433694.0000 - fp0.3: 4804465.0000 - tn0.3: 61868972.0000 - fn0.3: 2536073.0000 - precision0.3: 0.6626 - recall0.3: 0.7881 - tp0.5: 8253529.0000 - fp0.5: 21841
43.0000 - tn0.5: 64489292.0000 - fn0.5: 3716238.0000 - precision0.5: 0.7907 - recall0.5: 0.6895 - tp0.7: 6893438.0000 - fp0.7: 964306.0000 - tn0.7: 65709120.0000 - fn0.7: 5076329.0000 - pr
ecision0.7: 0.8773 - recall0.7: 0.5759 - tp0.9: 4825582.0000 - fp0.9: 253763.0000 - tn0.9: 66419652.0000 - fn0.9: 7144185.0000 - precision0.9: 0.9500 - recall0.9: 0.4031 - accuracy: 0.9250
 - auc: 0.9112 - f1: 0.2642 - val_loss: 0.7660 - val_tp0.1: 2600896.0000 - val_fp0.1: 1563848.0000 - val_tn0.1: 14934098.0000 - val_fn0.1: 561958.0000 - val_precision0.1: 0.6245 - val_reca
ll0.1: 0.8223 - val_tp0.3: 2498170.0000 - val_fp0.3: 1105437.0000 - val_tn0.3: 15392509.0000 - val_fn0.3: 664684.0000 - val_precision0.3: 0.6932 - val_recall0.3: 0.7898 - val_tp0.5: 236736
0.0000 - val_fp0.5: 720936.0000 - val_tn0.5: 15777010.0000 - val_fn0.5: 795494.0000 - val_precision0.5: 0.7666 - val_recall0.5: 0.7485 - val_tp0.7: 2247425.0000 - val_fp0.7: 512282.0000 -
val_tn0.7: 15985664.0000 - val_fn0.7: 915429.0000 - val_precision0.7: 0.8144 - val_recall0.7: 0.7106 - val_tp0.9: 2064899.0000 - val_fp0.9: 341222.0000 - val_tn0.9: 16156724.0000 - val_fn0
.9: 1097955.0000 - val_precision0.9: 0.8582 - val_recall0.9: 0.6529 - val_accuracy: 0.9229 - val_auc: 0.8919 - val_f1: 0.2772
Epoch 3/20
240/240 [==============================] - 123s 514ms/step - loss: 0.5296 - tp0.1: 10710507.0000 - fp0.1: 9627528.0000 - tn0.1: 57045904.0000 - fn0.1: 1259260.0000 - precision0.1: 0.5266 -
 recall0.1: 0.8948 - tp0.3: 9708967.0000 - fp0.3: 3890062.0000 - tn0.3: 62783372.0000 - fn0.3: 2260800.0000 - precision0.3: 0.7139 - recall0.3: 0.8111 - tp0.5: 8800137.0000 - fp0.5: 193587
4.0000 - tn0.5: 64737520.0000 - fn0.5: 3169630.0000 - precision0.5: 0.8197 - recall0.5: 0.7352 - tp0.7: 7616621.0000 - fp0.7: 897975.0000 - tn0.7: 65775424.0000 - fn0.7: 4353146.0000 - pre
cision0.7: 0.8945 - recall0.7: 0.6363 - tp0.9: 5653966.0000 - fp0.9: 272229.0000 - tn0.9: 66401216.0000 - fn0.9: 6315801.0000 - precision0.9: 0.9541 - recall0.9: 0.4724 - accuracy: 0.9351
- auc: 0.9281 - f1: 0.2642 - val_loss: 0.6681 - val_tp0.1: 2593120.0000 - val_fp0.1: 1019904.0000 - val_tn0.1: 15478042.0000 - val_fn0.1: 569734.0000 - val_precision0.1: 0.7177 - val_recal
l0.1: 0.8199 - val_tp0.3: 2244784.0000 - val_fp0.3: 338835.0000 - val_tn0.3: 16159111.0000 - val_fn0.3: 918070.0000 - val_precision0.3: 0.8689 - val_recall0.3: 0.7097 - val_tp0.5: 2014697.
0000 - val_fp0.5: 180709.0000 - val_tn0.5: 16317237.0000 - val_fn0.5: 1148157.0000 - val_precision0.5: 0.9177 - val_recall0.5: 0.6370 - val_tp0.7: 1651445.0000 - val_fp0.7: 66392.0000 - va
l_tn0.7: 16431554.0000 - val_fn0.7: 1511409.0000 - val_precision0.7: 0.9614 - val_recall0.7: 0.5221 - val_tp0.9: 1122584.0000 - val_fp0.9: 10626.0000 - val_tn0.9: 16487320.0000 - val_fn0.9
: 2040270.0000 - val_precision0.9: 0.9906 - val_recall0.9: 0.3549 - val_accuracy: 0.9324 - val_auc: 0.9151 - val_f1: 0.2772
Epoch 4/20
240/240 [==============================] - 123s 514ms/step - loss: 0.4720 - tp0.1: 10877075.0000 - fp0.1: 8671019.0000 - tn0.1: 58002392.0000 - fn0.1: 1092692.0000 - precision0.1: 0.5564 -
 recall0.1: 0.9087 - tp0.3: 9936344.0000 - fp0.3: 3411676.0000 - tn0.3: 63261736.0000 - fn0.3: 2033423.0000 - precision0.3: 0.7444 - recall0.3: 0.8301 - tp0.5: 9211814.0000 - fp0.5: 187395
8.0000 - tn0.5: 64799456.0000 - fn0.5: 2757953.0000 - precision0.5: 0.8310 - recall0.5: 0.7696 - tp0.7: 8139780.0000 - fp0.7: 898839.0000 - tn0.7: 65774596.0000 - fn0.7: 3829987.0000 - pre
cision0.7: 0.9006 - recall0.7: 0.6800 - tp0.9: 6238056.0000 - fp0.9: 255726.0000 - tn0.9: 66417696.0000 - fn0.9: 5731711.0000 - precision0.9: 0.9606 - recall0.9: 0.5212 - accuracy: 0.9411
- auc: 0.9384 - f1: 0.2642 - val_loss: 0.4134 - val_tp0.1: 3012908.0000 - val_fp0.1: 2692890.0000 - val_tn0.1: 13805056.0000 - val_fn0.1: 149946.0000 - val_precision0.1: 0.5280 - val_recal
l0.1: 0.9526 - val_tp0.3: 2844859.0000 - val_fp0.3: 1085275.0000 - val_tn0.3: 15412671.0000 - val_fn0.3: 317995.0000 - val_precision0.3: 0.7239 - val_recall0.3: 0.8995 - val_tp0.5: 2702953
.0000 - val_fp0.5: 596686.0000 - val_tn0.5: 15901260.0000 - val_fn0.5: 459901.0000 - val_precision0.5: 0.8192 - val_recall0.5: 0.8546 - val_tp0.7: 2523224.0000 - val_fp0.7: 328369.0000 - v
al_tn0.7: 16169577.0000 - val_fn0.7: 639630.0000 - val_precision0.7: 0.8848 - val_recall0.7: 0.7978 - val_tp0.9: 2192428.0000 - val_fp0.9: 138765.0000 - val_tn0.9: 16359181.0000 - val_fn0.
9: 970426.0000 - val_precision0.9: 0.9405 - val_recall0.9: 0.6932 - val_accuracy: 0.9463 - val_auc: 0.9625 - val_f1: 0.2772
Epoch 5/20
240/240 [==============================] - 124s 515ms/step - loss: 0.4310 - tp0.1: 10975989.0000 - fp0.1: 7934457.0000 - tn0.1: 58738992.0000 - fn0.1: 993778.0000 - precision0.1: 0.5804 -
recall0.1: 0.9170 - tp0.3: 10114842.0000 - fp0.3: 3195737.0000 - tn0.3: 63477704.0000 - fn0.3: 1854925.0000 - precision0.3: 0.7599 - recall0.3: 0.8450 - tp0.5: 9339053.0000 - fp0.5: 161448
5.0000 - tn0.5: 65058948.0000 - fn0.5: 2630714.0000 - precision0.5: 0.8526 - recall0.5: 0.7802 - tp0.7: 8449558.0000 - fp0.7: 805407.0000 - tn0.7: 65868032.0000 - fn0.7: 3520209.0000 - pre
cision0.7: 0.9130 - recall0.7: 0.7059 - tp0.9: 6730077.0000 - fp0.9: 232993.0000 - tn0.9: 66440428.0000 - fn0.9: 5239690.0000 - precision0.9: 0.9665 - recall0.9: 0.5623 - accuracy: 0.9460
- auc: 0.9454 - f1: 0.2642 - val_loss: 0.4139 - val_tp0.1: 3048945.0000 - val_fp0.1: 2758234.0000 - val_tn0.1: 13739712.0000 - val_fn0.1: 113909.0000 - val_precision0.1: 0.5250 - val_recal
l0.1: 0.9640 - val_tp0.3: 2947363.0000 - val_fp0.3: 1444749.0000 - val_tn0.3: 15053197.0000 - val_fn0.3: 215491.0000 - val_precision0.3: 0.6711 - val_recall0.3: 0.9319 - val_tp0.5: 2843052
.0000 - val_fp0.5: 901333.0000 - val_tn0.5: 15596613.0000 - val_fn0.5: 319802.0000 - val_precision0.5: 0.7593 - val_recall0.5: 0.8989 - val_tp0.7: 2732073.0000 - val_fp0.7: 569323.0000 - v
al_tn0.7: 15928623.0000 - val_fn0.7: 430781.0000 - val_precision0.7: 0.8276 - val_recall0.7: 0.8638 - val_tp0.9: 2540735.0000 - val_fp0.9: 283228.0000 - val_tn0.9: 16214718.0000 - val_fn0.
9: 622119.0000 - val_precision0.9: 0.8997 - val_recall0.9: 0.8033 - val_accuracy: 0.9379 - val_auc: 0.9680 - val_f1: 0.2772
Epoch 6/20
240/240 [==============================] - 123s 514ms/step - loss: 0.4055 - tp0.1: 11005938.0000 - fp0.1: 7117253.0000 - tn0.1: 59556168.0000 - fn0.1: 963829.0000 - precision0.1: 0.6073 -
recall0.1: 0.9195 - tp0.3: 10252977.0000 - fp0.3: 2993472.0000 - tn0.3: 63679960.0000 - fn0.3: 1716790.0000 - precision0.3: 0.7740 - recall0.3: 0.8566 - tp0.5: 9600067.0000 - fp0.5: 159785
2.0000 - tn0.5: 65075568.0000 - fn0.5: 2369700.0000 - precision0.5: 0.8573 - recall0.5: 0.8020 - tp0.7: 8735547.0000 - fp0.7: 775018.0000 - tn0.7: 65898436.0000 - fn0.7: 3234220.0000 - pre
cision0.7: 0.9185 - recall0.7: 0.7298 - tp0.9: 7088496.0000 - fp0.9: 233616.0000 - tn0.9: 66439832.0000 - fn0.9: 4881271.0000 - precision0.9: 0.9681 - recall0.9: 0.5922 - accuracy: 0.9496
- auc: 0.9484 - f1: 0.2642 - val_loss: 0.3925 - val_tp0.1: 3016157.0000 - val_fp0.1: 2494175.0000 - val_tn0.1: 14003771.0000 - val_fn0.1: 146697.0000 - val_precision0.1: 0.5474 - val_recal
l0.1: 0.9536 - val_tp0.3: 2913629.0000 - val_fp0.3: 1206347.0000 - val_tn0.3: 15291599.0000 - val_fn0.3: 249225.0000 - val_precision0.3: 0.7072 - val_recall0.3: 0.9212 - val_tp0.5: 2794468
.0000 - val_fp0.5: 707843.0000 - val_tn0.5: 15790103.0000 - val_fn0.5: 368386.0000 - val_precision0.5: 0.7979 - val_recall0.5: 0.8835 - val_tp0.7: 2631745.0000 - val_fp0.7: 409739.0000 - v
al_tn0.7: 16088207.0000 - val_fn0.7: 531109.0000 - val_precision0.7: 0.8653 - val_recall0.7: 0.8321 - val_tp0.9: 2298554.0000 - val_fp0.9: 168326.0000 - val_tn0.9: 16329620.0000 - val_fn0.
9: 864300.0000 - val_precision0.9: 0.9318 - val_recall0.9: 0.7267 - val_accuracy: 0.9453 - val_auc: 0.9633 - val_f1: 0.2772
Epoch 7/20
240/240 [==============================] - 123s 515ms/step - loss: 0.3854 - tp0.1: 11113311.0000 - fp0.1: 7284589.0000 - tn0.1: 59388840.0000 - fn0.1: 856456.0000 - precision0.1: 0.6041 -
recall0.1: 0.9284 - tp0.3: 10356441.0000 - fp0.3: 3026811.0000 - tn0.3: 63646640.0000 - fn0.3: 1613326.0000 - precision0.3: 0.7738 - recall0.3: 0.8652 - tp0.5: 9718595.0000 - fp0.5: 163300
4.0000 - tn0.5: 65040420.0000 - fn0.5: 2251172.0000 - precision0.5: 0.8561 - recall0.5: 0.8119 - tp0.7: 8835263.0000 - fp0.7: 784778.0000 - tn0.7: 65888660.0000 - fn0.7: 3134504.0000 - pre
cision0.7: 0.9184 - recall0.7: 0.7381 - tp0.9: 7228050.0000 - fp0.9: 242511.0000 - tn0.9: 66430912.0000 - fn0.9: 4741717.0000 - precision0.9: 0.9675 - recall0.9: 0.6039 - accuracy: 0.9506
- auc: 0.9535 - f1: 0.2642 - val_loss: 0.4443 - val_tp0.1: 2906183.0000 - val_fp0.1: 1626644.0000 - val_tn0.1: 14871302.0000 - val_fn0.1: 256671.0000 - val_precision0.1: 0.6411 - val_recal
l0.1: 0.9188 - val_tp0.3: 2816271.0000 - val_fp0.3: 1160666.0000 - val_tn0.3: 15337280.0000 - val_fn0.3: 346583.0000 - val_precision0.3: 0.7082 - val_recall0.3: 0.8904 - val_tp0.5: 2689144
.0000 - val_fp0.5: 809812.0000 - val_tn0.5: 15688134.0000 - val_fn0.5: 473710.0000 - val_precision0.5: 0.7686 - val_recall0.5: 0.8502 - val_tp0.7: 2488156.0000 - val_fp0.7: 481300.0000 - v
al_tn0.7: 16016646.0000 - val_fn0.7: 674698.0000 - val_precision0.7: 0.8379 - val_recall0.7: 0.7867 - val_tp0.9: 2127831.0000 - val_fp0.9: 230399.0000 - val_tn0.9: 16267547.0000 - val_fn0.
9: 1035023.0000 - val_precision0.9: 0.9023 - val_recall0.9: 0.6728 - val_accuracy: 0.9347 - val_auc: 0.9452 - val_f1: 0.2772
Epoch 8/20
240/240 [==============================] - 124s 515ms/step - loss: 0.3596 - tp0.1: 11147835.0000 - fp0.1: 6521359.0000 - tn0.1: 60152048.0000 - fn0.1: 821932.0000 - precision0.1: 0.6309 -
recall0.1: 0.9313 - tp0.3: 10458403.0000 - fp0.3: 2774423.0000 - tn0.3: 63899012.0000 - fn0.3: 1511364.0000 - precision0.3: 0.7903 - recall0.3: 0.8737 - tp0.5: 9862220.0000 - fp0.5: 151598
7.0000 - tn0.5: 65157436.0000 - fn0.5: 2107547.0000 - precision0.5: 0.8668 - recall0.5: 0.8239 - tp0.7: 9063431.0000 - fp0.7: 749399.0000 - tn0.7: 65924016.0000 - fn0.7: 2906336.0000 - pre
cision0.7: 0.9236 - recall0.7: 0.7572 - tp0.9: 7504772.0000 - fp0.9: 226379.0000 - tn0.9: 66447060.0000 - fn0.9: 4464995.0000 - precision0.9: 0.9707 - recall0.9: 0.6270 - accuracy: 0.9539
- auc: 0.9563 - f1: 0.2642 - val_loss: 0.5498 - val_tp0.1: 3055124.0000 - val_fp0.1: 4118468.0000 - val_tn0.1: 12379478.0000 - val_fn0.1: 107730.0000 - val_precision0.1: 0.4259 - val_recal
l0.1: 0.9659 - val_tp0.3: 2939354.0000 - val_fp0.3: 2257326.0000 - val_tn0.3: 14240620.0000 - val_fn0.3: 223500.0000 - val_precision0.3: 0.5656 - val_recall0.3: 0.9293 - val_tp0.5: 2829932
.0000 - val_fp0.5: 1526939.0000 - val_tn0.5: 14971007.0000 - val_fn0.5: 332922.0000 - val_precision0.5: 0.6495 - val_recall0.5: 0.8947 - val_tp0.7: 2642371.0000 - val_fp0.7: 901220.0000 -
val_tn0.7: 15596726.0000 - val_fn0.7: 520483.0000 - val_precision0.7: 0.7457 - val_recall0.7: 0.8354 - val_tp0.9: 2236370.0000 - val_fp0.9: 400953.0000 - val_tn0.9: 16096993.0000 - val_fn0
.9: 926484.0000 - val_precision0.9: 0.8480 - val_recall0.9: 0.7071 - val_accuracy: 0.9054 - val_auc: 0.9561 - val_f1: 0.2772
Epoch 9/20
240/240 [==============================] - 123s 514ms/step - loss: 0.3587 - tp0.1: 11154292.0000 - fp0.1: 6547650.0000 - tn0.1: 60125796.0000 - fn0.1: 815475.0000 - precision0.1: 0.6301 -
recall0.1: 0.9319 - tp0.3: 10488369.0000 - fp0.3: 2868809.0000 - tn0.3: 63804604.0000 - fn0.3: 1481398.0000 - precision0.3: 0.7852 - recall0.3: 0.8762 - tp0.5: 9875989.0000 - fp0.5: 153301
6.0000 - tn0.5: 65140432.0000 - fn0.5: 2093778.0000 - precision0.5: 0.8656 - recall0.5: 0.8251 - tp0.7: 9098782.0000 - fp0.7: 781237.0000 - tn0.7: 65892200.0000 - fn0.7: 2870985.0000 - pre
cision0.7: 0.9209 - recall0.7: 0.7601 - tp0.9: 7462273.0000 - fp0.9: 220634.0000 - tn0.9: 66452784.0000 - fn0.9: 4507494.0000 - precision0.9: 0.9713 - recall0.9: 0.6234 - accuracy: 0.9539
- auc: 0.9568 - f1: 0.2642 - val_loss: 0.3098 - val_tp0.1: 3044237.0000 - val_fp0.1: 1845158.0000 - val_tn0.1: 14652788.0000 - val_fn0.1: 118617.0000 - val_precision0.1: 0.6226 - val_recal
l0.1: 0.9625 - val_tp0.3: 2952931.0000 - val_fp0.3: 1010870.0000 - val_tn0.3: 15487076.0000 - val_fn0.3: 209923.0000 - val_precision0.3: 0.7450 - val_recall0.3: 0.9336 - val_tp0.5: 2818007
.0000 - val_fp0.5: 560464.0000 - val_tn0.5: 15937482.0000 - val_fn0.5: 344847.0000 - val_precision0.5: 0.8341 - val_recall0.5: 0.8910 - val_tp0.7: 2675497.0000 - val_fp0.7: 330511.0000 - v
al_tn0.7: 16167435.0000 - val_fn0.7: 487357.0000 - val_precision0.7: 0.8900 - val_recall0.7: 0.8459 - val_tp0.9: 2356811.0000 - val_fp0.9: 121488.0000 - val_tn0.9: 16376458.0000 - val_fn0.
9: 806043.0000 - val_precision0.9: 0.9510 - val_recall0.9: 0.7452 - val_accuracy: 0.9540 - val_auc: 0.9720 - val_f1: 0.2772
Epoch 10/20
240/240 [==============================] - 123s 514ms/step - loss: 0.3310 - tp0.1: 11220800.0000 - fp0.1: 6024418.0000 - tn0.1: 60649028.0000 - fn0.1: 748967.0000 - precision0.1: 0.6507 -
recall0.1: 0.9374 - tp0.3: 10602011.0000 - fp0.3: 2619415.0000 - tn0.3: 64054032.0000 - fn0.3: 1367756.0000 - precision0.3: 0.8019 - recall0.3: 0.8857 - tp0.5: 10035760.0000 - fp0.5: 14108
57.0000 - tn0.5: 65262568.0000 - fn0.5: 1934007.0000 - precision0.5: 0.8767 - recall0.5: 0.8384 - tp0.7: 9310399.0000 - fp0.7: 714073.0000 - tn0.7: 65959376.0000 - fn0.7: 2659368.0000 - pr
ecision0.7: 0.9288 - recall0.7: 0.7778 - tp0.9: 7802441.0000 - fp0.9: 197965.0000 - tn0.9: 66475472.0000 - fn0.9: 4167326.0000 - precision0.9: 0.9753 - recall0.9: 0.6518 - accuracy: 0.9575
 - auc: 0.9608 - f1: 0.2642 - val_loss: 0.3701 - val_tp0.1: 3063278.0000 - val_fp0.1: 2656513.0000 - val_tn0.1: 13841433.0000 - val_fn0.1: 99576.0000 - val_precision0.1: 0.5356 - val_recal
l0.1: 0.9685 - val_tp0.3: 2959133.0000 - val_fp0.3: 1261902.0000 - val_tn0.3: 15236044.0000 - val_fn0.3: 203721.0000 - val_precision0.3: 0.7010 - val_recall0.3: 0.9356 - val_tp0.5: 2866997
.0000 - val_fp0.5: 802391.0000 - val_tn0.5: 15695555.0000 - val_fn0.5: 295857.0000 - val_precision0.5: 0.7813 - val_recall0.5: 0.9065 - val_tp0.7: 2730525.0000 - val_fp0.7: 462170.0000 - v
al_tn0.7: 16035776.0000 - val_fn0.7: 432329.0000 - val_precision0.7: 0.8552 - val_recall0.7: 0.8633 - val_tp0.9: 2464570.0000 - val_fp0.9: 189700.0000 - val_tn0.9: 16308246.0000 - val_fn0.
9: 698284.0000 - val_precision0.9: 0.9285 - val_recall0.9: 0.7792 - val_accuracy: 0.9441 - val_auc: 0.9728 - val_f1: 0.2772
Epoch 11/20
240/240 [==============================] - 123s 514ms/step - loss: 0.3233 - tp0.1: 11231645.0000 - fp0.1: 5931503.0000 - tn0.1: 60741932.0000 - fn0.1: 738122.0000 - precision0.1: 0.6544 -
recall0.1: 0.9383 - tp0.3: 10598514.0000 - fp0.3: 2444604.0000 - tn0.3: 64228848.0000 - fn0.3: 1371253.0000 - precision0.3: 0.8126 - recall0.3: 0.8854 - tp0.5: 10075292.0000 - fp0.5: 13321
40.0000 - tn0.5: 65341256.0000 - fn0.5: 1894475.0000 - precision0.5: 0.8832 - recall0.5: 0.8417 - tp0.7: 9378243.0000 - fp0.7: 674133.0000 - tn0.7: 65999280.0000 - fn0.7: 2591524.0000 - pr
ecision0.7: 0.9329 - recall0.7: 0.7835 - tp0.9: 7966249.0000 - fp0.9: 193209.0000 - tn0.9: 66480236.0000 - fn0.9: 4003518.0000 - precision0.9: 0.9763 - recall0.9: 0.6655 - accuracy: 0.9590
 - auc: 0.9619 - f1: 0.2642 - val_loss: 0.2986 - val_tp0.1: 3049223.0000 - val_fp0.1: 1879104.0000 - val_tn0.1: 14618842.0000 - val_fn0.1: 113631.0000 - val_precision0.1: 0.6187 - val_reca
ll0.1: 0.9641 - val_tp0.3: 2957119.0000 - val_fp0.3: 970547.0000 - val_tn0.3: 15527399.0000 - val_fn0.3: 205735.0000 - val_precision0.3: 0.7529 - val_recall0.3: 0.9350 - val_tp0.5: 2848369
.0000 - val_fp0.5: 566365.0000 - val_tn0.5: 15931581.0000 - val_fn0.5: 314485.0000 - val_precision0.5: 0.8341 - val_recall0.5: 0.9006 - val_tp0.7: 2684500.0000 - val_fp0.7: 298668.0000 - v
al_tn0.7: 16199278.0000 - val_fn0.7: 478354.0000 - val_precision0.7: 0.8999 - val_recall0.7: 0.8488 - val_tp0.9: 2264810.0000 - val_fp0.9: 80756.0000 - val_tn0.9: 16417190.0000 - val_fn0.9
: 898044.0000 - val_precision0.9: 0.9656 - val_recall0.9: 0.7161 - val_accuracy: 0.9552 - val_auc: 0.9738 - val_f1: 0.2772
Epoch 12/20
240/240 [==============================] - 123s 514ms/step - loss: 0.3085 - tp0.1: 11270470.0000 - fp0.1: 5619369.0000 - tn0.1: 61054068.0000 - fn0.1: 699297.0000 - precision0.1: 0.6673 -
recall0.1: 0.9416 - tp0.3: 10714167.0000 - fp0.3: 2487268.0000 - tn0.3: 64186176.0000 - fn0.3: 1255600.0000 - precision0.3: 0.8116 - recall0.3: 0.8951 - tp0.5: 10175108.0000 - fp0.5: 13504
36.0000 - tn0.5: 65323020.0000 - fn0.5: 1794659.0000 - precision0.5: 0.8828 - recall0.5: 0.8501 - tp0.7: 9471869.0000 - fp0.7: 668828.0000 - tn0.7: 66004600.0000 - fn0.7: 2497898.0000 - pr
ecision0.7: 0.9340 - recall0.7: 0.7913 - tp0.9: 8087818.0000 - fp0.9: 195376.0000 - tn0.9: 66478052.0000 - fn0.9: 3881949.0000 - precision0.9: 0.9764 - recall0.9: 0.6757 - accuracy: 0.9600
 - auc: 0.9640 - f1: 0.2642 - val_loss: 0.3350 - val_tp0.1: 3080571.0000 - val_fp0.1: 2712529.0000 - val_tn0.1: 13785417.0000 - val_fn0.1: 82283.0000 - val_precision0.1: 0.5318 - val_recal
l0.1: 0.9740 - val_tp0.3: 2997438.0000 - val_fp0.3: 1233176.0000 - val_tn0.3: 15264770.0000 - val_fn0.3: 165416.0000 - val_precision0.3: 0.7085 - val_recall0.3: 0.9477 - val_tp0.5: 2901164
.0000 - val_fp0.5: 678226.0000 - val_tn0.5: 15819720.0000 - val_fn0.5: 261690.0000 - val_precision0.5: 0.8105 - val_recall0.5: 0.9173 - val_tp0.7: 2761496.0000 - val_fp0.7: 360082.0000 - v
al_tn0.7: 16137864.0000 - val_fn0.7: 401358.0000 - val_precision0.7: 0.8846 - val_recall0.7: 0.8731 - val_tp0.9: 2425682.0000 - val_fp0.9: 106600.0000 - val_tn0.9: 16391346.0000 - val_fn0.
9: 737172.0000 - val_precision0.9: 0.9579 - val_recall0.9: 0.7669 - val_accuracy: 0.9522 - val_auc: 0.9775 - val_f1: 0.2772
Epoch 13/20
240/240 [==============================] - 123s 513ms/step - loss: 0.2998 - tp0.1: 11289427.0000 - fp0.1: 5435141.0000 - tn0.1: 61238288.0000 - fn0.1: 680340.0000 - precision0.1: 0.6750 -
recall0.1: 0.9432 - tp0.3: 10775991.0000 - fp0.3: 2541120.0000 - tn0.3: 64132324.0000 - fn0.3: 1193776.0000 - precision0.3: 0.8092 - recall0.3: 0.9003 - tp0.5: 10236179.0000 - fp0.5: 13511
65.0000 - tn0.5: 65322272.0000 - fn0.5: 1733588.0000 - precision0.5: 0.8834 - recall0.5: 0.8552 - tp0.7: 9552300.0000 - fp0.7: 667311.0000 - tn0.7: 66006116.0000 - fn0.7: 2417467.0000 - pr
ecision0.7: 0.9347 - recall0.7: 0.7980 - tp0.9: 8197249.0000 - fp0.9: 192011.0000 - tn0.9: 66481424.0000 - fn0.9: 3772518.0000 - precision0.9: 0.9771 - recall0.9: 0.6848 - accuracy: 0.9608
 - auc: 0.9652 - f1: 0.2642 - val_loss: 0.2859 - val_tp0.1: 2992948.0000 - val_fp0.1: 1104053.0000 - val_tn0.1: 15393893.0000 - val_fn0.1: 169906.0000 - val_precision0.1: 0.7305 - val_reca
ll0.1: 0.9463 - val_tp0.3: 2859768.0000 - val_fp0.3: 511820.0000 - val_tn0.3: 15986126.0000 - val_fn0.3: 303086.0000 - val_precision0.3: 0.8482 - val_recall0.3: 0.9042 - val_tp0.5: 2679052
.0000 - val_fp0.5: 241764.0000 - val_tn0.5: 16256182.0000 - val_fn0.5: 483802.0000 - val_precision0.5: 0.9172 - val_recall0.5: 0.8470 - val_tp0.7: 2445930.0000 - val_fp0.7: 113353.0000 - v
al_tn0.7: 16384593.0000 - val_fn0.7: 716924.0000 - val_precision0.7: 0.9557 - val_recall0.7: 0.7733 - val_tp0.9: 2024394.0000 - val_fp0.9: 30607.0000 - val_tn0.9: 16467339.0000 - val_fn0.9
: 1138460.0000 - val_precision0.9: 0.9851 - val_recall0.9: 0.6401 - val_accuracy: 0.9631 - val_auc: 0.9677 - val_f1: 0.2772
Epoch 14/20
240/240 [==============================] - 123s 513ms/step - loss: 0.2873 - tp0.1: 11325492.0000 - fp0.1: 5158653.0000 - tn0.1: 61514784.0000 - fn0.1: 644275.0000 - precision0.1: 0.6871 -
recall0.1: 0.9462 - tp0.3: 10839018.0000 - fp0.3: 2435459.0000 - tn0.3: 64237968.0000 - fn0.3: 1130749.0000 - precision0.3: 0.8165 - recall0.3: 0.9055 - tp0.5: 10316818.0000 - fp0.5: 13405
37.0000 - tn0.5: 65332924.0000 - fn0.5: 1652949.0000 - precision0.5: 0.8850 - recall0.5: 0.8619 - tp0.7: 9615679.0000 - fp0.7: 663572.0000 - tn0.7: 66009872.0000 - fn0.7: 2354088.0000 - pr
ecision0.7: 0.9354 - recall0.7: 0.8033 - tp0.9: 8226896.0000 - fp0.9: 190675.0000 - tn0.9: 66482748.0000 - fn0.9: 3742871.0000 - precision0.9: 0.9773 - recall0.9: 0.6873 - accuracy: 0.9619
 - auc: 0.9668 - f1: 0.2642 - val_loss: 0.2843 - val_tp0.1: 3018528.0000 - val_fp0.1: 1341368.0000 - val_tn0.1: 15156578.0000 - val_fn0.1: 144326.0000 - val_precision0.1: 0.6923 - val_reca
ll0.1: 0.9544 - val_tp0.3: 2884535.0000 - val_fp0.3: 604165.0000 - val_tn0.3: 15893781.0000 - val_fn0.3: 278319.0000 - val_precision0.3: 0.8268 - val_recall0.3: 0.9120 - val_tp0.5: 2750904
.0000 - val_fp0.5: 350090.0000 - val_tn0.5: 16147856.0000 - val_fn0.5: 411950.0000 - val_precision0.5: 0.8871 - val_recall0.5: 0.8698 - val_tp0.7: 2579988.0000 - val_fp0.7: 191060.0000 - v
al_tn0.7: 16306886.0000 - val_fn0.7: 582866.0000 - val_precision0.7: 0.9311 - val_recall0.7: 0.8157 - val_tp0.9: 2263427.0000 - val_fp0.9: 74463.0000 - val_tn0.9: 16423483.0000 - val_fn0.9
: 899427.0000 - val_precision0.9: 0.9681 - val_recall0.9: 0.7156 - val_accuracy: 0.9612 - val_auc: 0.9707 - val_f1: 0.2772
Epoch 15/20
240/240 [==============================] - 123s 513ms/step - loss: 0.2841 - tp0.1: 11339440.0000 - fp0.1: 5281029.0000 - tn0.1: 61392416.0000 - fn0.1: 630327.0000 - precision0.1: 0.6823 -
recall0.1: 0.9473 - tp0.3: 10822522.0000 - fp0.3: 2360182.0000 - tn0.3: 64313252.0000 - fn0.3: 1147245.0000 - precision0.3: 0.8210 - recall0.3: 0.9042 - tp0.5: 10312399.0000 - fp0.5: 12761
78.0000 - tn0.5: 65397248.0000 - fn0.5: 1657368.0000 - precision0.5: 0.8899 - recall0.5: 0.8615 - tp0.7: 9644338.0000 - fp0.7: 625693.0000 - tn0.7: 66047756.0000 - fn0.7: 2325429.0000 - pr
ecision0.7: 0.9391 - recall0.7: 0.8057 - tp0.9: 8361871.0000 - fp0.9: 174964.0000 - tn0.9: 66498432.0000 - fn0.9: 3607896.0000 - precision0.9: 0.9795 - recall0.9: 0.6986 - accuracy: 0.9627
 - auc: 0.9678 - f1: 0.2642 - val_loss: 0.2785 - val_tp0.1: 3071530.0000 - val_fp0.1: 1940363.0000 - val_tn0.1: 14557583.0000 - val_fn0.1: 91324.0000 - val_precision0.1: 0.6128 - val_recal
l0.1: 0.9711 - val_tp0.3: 2963120.0000 - val_fp0.3: 829784.0000 - val_tn0.3: 15668162.0000 - val_fn0.3: 199734.0000 - val_precision0.3: 0.7812 - val_recall0.3: 0.9369 - val_tp0.5: 2835343.
0000 - val_fp0.5: 408144.0000 - val_tn0.5: 16089802.0000 - val_fn0.5: 327511.0000 - val_precision0.5: 0.8742 - val_recall0.5: 0.8965 - val_tp0.7: 2617565.0000 - val_fp0.7: 178411.0000 - va
l_tn0.7: 16319535.0000 - val_fn0.7: 545289.0000 - val_precision0.7: 0.9362 - val_recall0.7: 0.8276 - val_tp0.9: 2212166.0000 - val_fp0.9: 48362.0000 - val_tn0.9: 16449584.0000 - val_fn0.9:
 950688.0000 - val_precision0.9: 0.9786 - val_recall0.9: 0.6994 - val_accuracy: 0.9626 - val_auc: 0.9783 - val_f1: 0.2772
Epoch 16/20
240/240 [==============================] - 123s 513ms/step - loss: 0.2733 - tp0.1: 11349878.0000 - fp0.1: 4939772.0000 - tn0.1: 61733644.0000 - fn0.1: 619889.0000 - precision0.1: 0.6968 -
recall0.1: 0.9482 - tp0.3: 10872946.0000 - fp0.3: 2277823.0000 - tn0.3: 64395616.0000 - fn0.3: 1096821.0000 - precision0.3: 0.8268 - recall0.3: 0.9084 - tp0.5: 10390972.0000 - fp0.5: 12472
64.0000 - tn0.5: 65426192.0000 - fn0.5: 1578795.0000 - precision0.5: 0.8928 - recall0.5: 0.8681 - tp0.7: 9736581.0000 - fp0.7: 597016.0000 - tn0.7: 66076408.0000 - fn0.7: 2233186.0000 - pr
ecision0.7: 0.9422 - recall0.7: 0.8134 - tp0.9: 8491483.0000 - fp0.9: 164275.0000 - tn0.9: 66509160.0000 - fn0.9: 3478284.0000 - precision0.9: 0.9810 - recall0.9: 0.7094 - accuracy: 0.9641
 - auc: 0.9686 - f1: 0.2642 - val_loss: 0.2719 - val_tp0.1: 3021839.0000 - val_fp0.1: 1339815.0000 - val_tn0.1: 15158131.0000 - val_fn0.1: 141015.0000 - val_precision0.1: 0.6928 - val_reca
ll0.1: 0.9554 - val_tp0.3: 2908072.0000 - val_fp0.3: 621879.0000 - val_tn0.3: 15876067.0000 - val_fn0.3: 254782.0000 - val_precision0.3: 0.8238 - val_recall0.3: 0.9194 - val_tp0.5: 2802958
.0000 - val_fp0.5: 363679.0000 - val_tn0.5: 16134267.0000 - val_fn0.5: 359896.0000 - val_precision0.5: 0.8852 - val_recall0.5: 0.8862 - val_tp0.7: 2647510.0000 - val_fp0.7: 188747.0000 - v
al_tn0.7: 16309199.0000 - val_fn0.7: 515344.0000 - val_precision0.7: 0.9335 - val_recall0.7: 0.8371 - val_tp0.9: 2292567.0000 - val_fp0.9: 53401.0000 - val_tn0.9: 16444545.0000 - val_fn0.9
: 870287.0000 - val_precision0.9: 0.9772 - val_recall0.9: 0.7248 - val_accuracy: 0.9632 - val_auc: 0.9717 - val_f1: 0.2772
Epoch 17/20
240/240 [==============================] - 123s 512ms/step - loss: 0.2657 - tp0.1: 11376204.0000 - fp0.1: 4892198.0000 - tn0.1: 61781224.0000 - fn0.1: 593563.0000 - precision0.1: 0.6993 -
recall0.1: 0.9504 - tp0.3: 10908750.0000 - fp0.3: 2245727.0000 - tn0.3: 64427696.0000 - fn0.3: 1061017.0000 - precision0.3: 0.8293 - recall0.3: 0.9114 - tp0.5: 10433698.0000 - fp0.5: 12316
34.0000 - tn0.5: 65441800.0000 - fn0.5: 1536069.0000 - precision0.5: 0.8944 - recall0.5: 0.8717 - tp0.7: 9778950.0000 - fp0.7: 598659.0000 - tn0.7: 66074788.0000 - fn0.7: 2190817.0000 - pr
ecision0.7: 0.9423 - recall0.7: 0.8170 - tp0.9: 8517790.0000 - fp0.9: 162128.0000 - tn0.9: 66511320.0000 - fn0.9: 3451977.0000 - precision0.9: 0.9813 - recall0.9: 0.7116 - accuracy: 0.9648
 - auc: 0.9699 - f1: 0.2642 - val_loss: 0.2613 - val_tp0.1: 3051492.0000 - val_fp0.1: 1505492.0000 - val_tn0.1: 14992454.0000 - val_fn0.1: 111362.0000 - val_precision0.1: 0.6696 - val_reca
ll0.1: 0.9648 - val_tp0.3: 2939774.0000 - val_fp0.3: 659951.0000 - val_tn0.3: 15837995.0000 - val_fn0.3: 223080.0000 - val_precision0.3: 0.8167 - val_recall0.3: 0.9295 - val_tp0.5: 2833524
.0000 - val_fp0.5: 395235.0000 - val_tn0.5: 16102711.0000 - val_fn0.5: 329330.0000 - val_precision0.5: 0.8776 - val_recall0.5: 0.8959 - val_tp0.7: 2680855.0000 - val_fp0.7: 215919.0000 - v
al_tn0.7: 16282027.0000 - val_fn0.7: 481999.0000 - val_precision0.7: 0.9255 - val_recall0.7: 0.8476 - val_tp0.9: 2359116.0000 - val_fp0.9: 75353.0000 - val_tn0.9: 16422593.0000 - val_fn0.9
: 803738.0000 - val_precision0.9: 0.9690 - val_recall0.9: 0.7459 - val_accuracy: 0.9631 - val_auc: 0.9765 - val_f1: 0.2772
Epoch 18/20
240/240 [==============================] - 123s 513ms/step - loss: 0.2598 - tp0.1: 11386263.0000 - fp0.1: 4766580.0000 - tn0.1: 61906832.0000 - fn0.1: 583504.0000 - precision0.1: 0.7049 -
recall0.1: 0.9513 - tp0.3: 10931522.0000 - fp0.3: 2185962.0000 - tn0.3: 64487472.0000 - fn0.3: 1038245.0000 - precision0.3: 0.8334 - recall0.3: 0.9133 - tp0.5: 10479377.0000 - fp0.5: 12310
07.0000 - tn0.5: 65442416.0000 - fn0.5: 1490390.0000 - precision0.5: 0.8949 - recall0.5: 0.8755 - tp0.7: 9829899.0000 - fp0.7: 592904.0000 - tn0.7: 66080536.0000 - fn0.7: 2139868.0000 - pr
ecision0.7: 0.9431 - recall0.7: 0.8212 - tp0.9: 8602261.0000 - fp0.9: 161222.0000 - tn0.9: 66512188.0000 - fn0.9: 3367506.0000 - precision0.9: 0.9816 - recall0.9: 0.7187 - accuracy: 0.9654
 - auc: 0.9706 - f1: 0.2642 - val_loss: 0.2608 - val_tp0.1: 3020760.0000 - val_fp0.1: 1219927.0000 - val_tn0.1: 15278019.0000 - val_fn0.1: 142094.0000 - val_precision0.1: 0.7123 - val_reca
ll0.1: 0.9551 - val_tp0.3: 2909302.0000 - val_fp0.3: 580392.0000 - val_tn0.3: 15917554.0000 - val_fn0.3: 253552.0000 - val_precision0.3: 0.8337 - val_recall0.3: 0.9198 - val_tp0.5: 2799018
.0000 - val_fp0.5: 344609.0000 - val_tn0.5: 16153337.0000 - val_fn0.5: 363836.0000 - val_precision0.5: 0.8904 - val_recall0.5: 0.8850 - val_tp0.7: 2633849.0000 - val_fp0.7: 176599.0000 - v
al_tn0.7: 16321347.0000 - val_fn0.7: 529005.0000 - val_precision0.7: 0.9372 - val_recall0.7: 0.8327 - val_tp0.9: 2302923.0000 - val_fp0.9: 54090.0000 - val_tn0.9: 16443856.0000 - val_fn0.9
: 859931.0000 - val_precision0.9: 0.9771 - val_recall0.9: 0.7281 - val_accuracy: 0.9640 - val_auc: 0.9722 - val_f1: 0.2772
Epoch 19/20
240/240 [==============================] - 123s 514ms/step - loss: 0.2516 - tp0.1: 11404546.0000 - fp0.1: 4602276.0000 - tn0.1: 62071184.0000 - fn0.1: 565221.0000 - precision0.1: 0.7125 -
recall0.1: 0.9528 - tp0.3: 10964712.0000 - fp0.3: 2131443.0000 - tn0.3: 64541984.0000 - fn0.3: 1005055.0000 - precision0.3: 0.8372 - recall0.3: 0.9160 - tp0.5: 10530069.0000 - fp0.5: 11977
48.0000 - tn0.5: 65475676.0000 - fn0.5: 1439698.0000 - precision0.5: 0.8979 - recall0.5: 0.8797 - tp0.7: 9887845.0000 - fp0.7: 573172.0000 - tn0.7: 66100272.0000 - fn0.7: 2081922.0000 - pr
ecision0.7: 0.9452 - recall0.7: 0.8261 - tp0.9: 8654050.0000 - fp0.9: 150095.0000 - tn0.9: 66523328.0000 - fn0.9: 3315717.0000 - precision0.9: 0.9830 - recall0.9: 0.7230 - accuracy: 0.9665
 - auc: 0.9717 - f1: 0.2642 - val_loss: 0.2481 - val_tp0.1: 3008011.0000 - val_fp0.1: 953689.0000 - val_tn0.1: 15544257.0000 - val_fn0.1: 154843.0000 - val_precision0.1: 0.7593 - val_recal
l0.1: 0.9510 - val_tp0.3: 2881653.0000 - val_fp0.3: 443165.0000 - val_tn0.3: 16054781.0000 - val_fn0.3: 281201.0000 - val_precision0.3: 0.8667 - val_recall0.3: 0.9111 - val_tp0.5: 2768505.
0000 - val_fp0.5: 264471.0000 - val_tn0.5: 16233475.0000 - val_fn0.5: 394349.0000 - val_precision0.5: 0.9128 - val_recall0.5: 0.8753 - val_tp0.7: 2606170.0000 - val_fp0.7: 136595.0000 - va
l_tn0.7: 16361351.0000 - val_fn0.7: 556684.0000 - val_precision0.7: 0.9502 - val_recall0.7: 0.8240 - val_tp0.9: 2281582.0000 - val_fp0.9: 42534.0000 - val_tn0.9: 16455412.0000 - val_fn0.9:
 881272.0000 - val_precision0.9: 0.9817 - val_recall0.9: 0.7214 - val_accuracy: 0.9665 - val_auc: 0.9714 - val_f1: 0.2772
Epoch 20/20
240/240 [==============================] - 123s 513ms/step - loss: 0.2463 - tp0.1: 11418792.0000 - fp0.1: 4532019.0000 - tn0.1: 62141424.0000 - fn0.1: 550975.0000 - precision0.1: 0.7159 -
recall0.1: 0.9540 - tp0.3: 10983398.0000 - fp0.3: 2088662.0000 - tn0.3: 64584768.0000 - fn0.3: 986369.0000 - precision0.3: 0.8402 - recall0.3: 0.9176 - tp0.5: 10555722.0000 - fp0.5: 118115
9.0000 - tn0.5: 65492296.0000 - fn0.5: 1414045.0000 - precision0.5: 0.8994 - recall0.5: 0.8819 - tp0.7: 9935653.0000 - fp0.7: 569774.0000 - tn0.7: 66103648.0000 - fn0.7: 2034114.0000 - pre
cision0.7: 0.9458 - recall0.7: 0.8301 - tp0.9: 8727770.0000 - fp0.9: 149122.0000 - tn0.9: 66524300.0000 - fn0.9: 3241997.0000 - precision0.9: 0.9832 - recall0.9: 0.7292 - accuracy: 0.9670
- auc: 0.9724 - f1: 0.2642 - val_loss: 0.2428 - val_tp0.1: 3043089.0000 - val_fp0.1: 1254723.0000 - val_tn0.1: 15243223.0000 - val_fn0.1: 119765.0000 - val_precision0.1: 0.7081 - val_recal
l0.1: 0.9621 - val_tp0.3: 2938943.0000 - val_fp0.3: 582622.0000 - val_tn0.3: 15915324.0000 - val_fn0.3: 223911.0000 - val_precision0.3: 0.8346 - val_recall0.3: 0.9292 - val_tp0.5: 2838207.
0000 - val_fp0.5: 352144.0000 - val_tn0.5: 16145802.0000 - val_fn0.5: 324647.0000 - val_precision0.5: 0.8896 - val_recall0.5: 0.8974 - val_tp0.7: 2688032.0000 - val_fp0.7: 185761.0000 - va
l_tn0.7: 16312185.0000 - val_fn0.7: 474822.0000 - val_precision0.7: 0.9354 - val_recall0.7: 0.8499 - val_tp0.9: 2371174.0000 - val_fp0.9: 57559.0000 - val_tn0.9: 16440387.0000 - val_fn0.9:
 791680.0000 - val_precision0.9: 0.9763 - val_recall0.9: 0.7497 - val_accuracy: 0.9656 - val_auc: 0.9761 - val_f1: 0.2772
--- Running training session 62/140
{'hp_epochs': 20, 'hp_batch_size': 20, 'hp_scaler': 'standard', 'hp_n_levels': 3, 'hp_first_filters': 128, 'hp_pool_size': 4, 'hp_input_size': 16384, 'hp_lr_start': 0.04354970735327304, 'h
p_lr_power': 1.0}
--- repeat #: 2
input - shape:   (None, 16384, 1)
output - shape:  (None, 16384, 1)
Epoch 1/20
240/240 [==============================] - 144s 534ms/step - loss: 1.1479 - tp0.1: 9906728.0000 - fp0.1: 13187183.0000 - tn0.1: 53486260.0000 - fn0.1: 2063039.0000 - precision0.1: 0.4290 -
 recall0.1: 0.8276 - tp0.3: 8671992.0000 - fp0.3: 5127913.0000 - tn0.3: 61545512.0000 - fn0.3: 3297775.0000 - precision0.3: 0.6284 - recall0.3: 0.7245 - tp0.5: 7568431.0000 - fp0.5: 253412
2.0000 - tn0.5: 64139300.0000 - fn0.5: 4401336.0000 - precision0.5: 0.7492 - recall0.5: 0.6323 - tp0.7: 6054670.0000 - fp0.7: 1016029.0000 - tn0.7: 65657384.0000 - fn0.7: 5915097.0000 - pr
ecision0.7: 0.8563 - recall0.7: 0.5058 - tp0.9: 3696361.0000 - fp0.9: 210116.0000 - tn0.9: 66463280.0000 - fn0.9: 8273406.0000 - precision0.9: 0.9462 - recall0.9: 0.3088 - accuracy: 0.9118
 - auc: 0.8839 - f1: 0.2642 - val_loss: 1.3309 - val_tp0.1: 1733257.0000 - val_fp0.1: 2884942.0000 - val_tn0.1: 13613004.0000 - val_fn0.1: 1429597.0000 - val_precision0.1: 0.3753 - val_rec
all0.1: 0.5480 - val_tp0.3: 1371992.0000 - val_fp0.3: 689902.0000 - val_tn0.3: 15808044.0000 - val_fn0.3: 1790862.0000 - val_precision0.3: 0.6654 - val_recall0.3: 0.4338 - val_tp0.5: 12399
93.0000 - val_fp0.5: 314549.0000 - val_tn0.5: 16183397.0000 - val_fn0.5: 1922861.0000 - val_precision0.5: 0.7977 - val_recall0.5: 0.3920 - val_tp0.7: 1088844.0000 - val_fp0.7: 172991.0000
- val_tn0.7: 16324955.0000 - val_fn0.7: 2074010.0000 - val_precision0.7: 0.8629 - val_recall0.7: 0.3443 - val_tp0.9: 854487.0000 - val_fp0.9: 85499.0000 - val_tn0.9: 16412447.0000 - val_fn
0.9: 2308367.0000 - val_precision0.9: 0.9090 - val_recall0.9: 0.2702 - val_accuracy: 0.8862 - val_auc: 0.7423 - val_f1: 0.2772
Epoch 2/20
240/240 [==============================] - 124s 516ms/step - loss: 0.8128 - tp0.1: 10386764.0000 - fp0.1: 10662276.0000 - tn0.1: 56011152.0000 - fn0.1: 1583003.0000 - precision0.1: 0.4935
- recall0.1: 0.8677 - tp0.3: 9374552.0000 - fp0.3: 4301798.0000 - tn0.3: 62371648.0000 - fn0.3: 2595215.0000 - precision0.3: 0.6855 - recall0.3: 0.7832 - tp0.5: 8455753.0000 - fp0.5: 22081
36.0000 - tn0.5: 64465300.0000 - fn0.5: 3514014.0000 - precision0.5: 0.7929 - recall0.5: 0.7064 - tp0.7: 7249754.0000 - fp0.7: 1037037.0000 - tn0.7: 65636376.0000 - fn0.7: 4720013.0000 - p
recision0.7: 0.8749 - recall0.7: 0.6057 - tp0.9: 4971364.0000 - fp0.9: 253682.0000 - tn0.9: 66419772.0000 - fn0.9: 6998403.0000 - precision0.9: 0.9514 - recall0.9: 0.4153 - accuracy: 0.927
2 - auc: 0.9156 - f1: 0.2642 - val_loss: 1.7318 - val_tp0.1: 1429651.0000 - val_fp0.1: 93149.0000 - val_tn0.1: 16404797.0000 - val_fn0.1: 1733203.0000 - val_precision0.1: 0.9388 - val_reca
ll0.1: 0.4520 - val_tp0.3: 1260333.0000 - val_fp0.3: 61973.0000 - val_tn0.3: 16435973.0000 - val_fn0.3: 1902521.0000 - val_precision0.3: 0.9531 - val_recall0.3: 0.3985 - val_tp0.5: 1164560
.0000 - val_fp0.5: 49155.0000 - val_tn0.5: 16448791.0000 - val_fn0.5: 1998294.0000 - val_precision0.5: 0.9595 - val_recall0.5: 0.3682 - val_tp0.7: 1052319.0000 - val_fp0.7: 36199.0000 - va
l_tn0.7: 16461747.0000 - val_fn0.7: 2110535.0000 - val_precision0.7: 0.9667 - val_recall0.7: 0.3327 - val_tp0.9: 812871.0000 - val_fp0.9: 16636.0000 - val_tn0.9: 16481310.0000 - val_fn0.9:
 2349983.0000 - val_precision0.9: 0.9799 - val_recall0.9: 0.2570 - val_accuracy: 0.8959 - val_auc: 0.7346 - val_f1: 0.2772
Epoch 3/20
240/240 [==============================] - 124s 516ms/step - loss: 0.5249 - tp0.1: 10650727.0000 - fp0.1: 8971663.0000 - tn0.1: 57701744.0000 - fn0.1: 1319040.0000 - precision0.1: 0.5428 -
 recall0.1: 0.8898 - tp0.3: 9707938.0000 - fp0.3: 3584621.0000 - tn0.3: 63088792.0000 - fn0.3: 2261829.0000 - precision0.3: 0.7303 - recall0.3: 0.8110 - tp0.5: 8947919.0000 - fp0.5: 196644
3.0000 - tn0.5: 64706976.0000 - fn0.5: 3021848.0000 - precision0.5: 0.8198 - recall0.5: 0.7475 - tp0.7: 7846298.0000 - fp0.7: 941117.0000 - tn0.7: 65732308.0000 - fn0.7: 4123469.0000 - pre
cision0.7: 0.8929 - recall0.7: 0.6555 - tp0.9: 5813133.0000 - fp0.9: 247129.0000 - tn0.9: 66426308.0000 - fn0.9: 6156634.0000 - precision0.9: 0.9592 - recall0.9: 0.4857 - accuracy: 0.9366
- auc: 0.9272 - f1: 0.2642 - val_loss: 0.6610 - val_tp0.1: 2455351.0000 - val_fp0.1: 660521.0000 - val_tn0.1: 15837425.0000 - val_fn0.1: 707503.0000 - val_precision0.1: 0.7880 - val_recall
0.1: 0.7763 - val_tp0.3: 2077241.0000 - val_fp0.3: 245933.0000 - val_tn0.3: 16252013.0000 - val_fn0.3: 1085613.0000 - val_precision0.3: 0.8941 - val_recall0.3: 0.6568 - val_tp0.5: 1903683.
0000 - val_fp0.5: 169535.0000 - val_tn0.5: 16328411.0000 - val_fn0.5: 1259171.0000 - val_precision0.5: 0.9182 - val_recall0.5: 0.6019 - val_tp0.7: 1686781.0000 - val_fp0.7: 108097.0000 - v
al_tn0.7: 16389849.0000 - val_fn0.7: 1476073.0000 - val_precision0.7: 0.9398 - val_recall0.7: 0.5333 - val_tp0.9: 1239931.0000 - val_fp0.9: 39608.0000 - val_tn0.9: 16458338.0000 - val_fn0.
9: 1922923.0000 - val_precision0.9: 0.9690 - val_recall0.9: 0.3920 - val_accuracy: 0.9273 - val_auc: 0.8923 - val_f1: 0.2772
Epoch 4/20
240/240 [==============================] - 124s 517ms/step - loss: 0.4701 - tp0.1: 10831550.0000 - fp0.1: 8197166.0000 - tn0.1: 58476288.0000 - fn0.1: 1138217.0000 - precision0.1: 0.5692 -
 recall0.1: 0.9049 - tp0.3: 10003164.0000 - fp0.3: 3493324.0000 - tn0.3: 63180092.0000 - fn0.3: 1966603.0000 - precision0.3: 0.7412 - recall0.3: 0.8357 - tp0.5: 9170982.0000 - fp0.5: 17747
67.0000 - tn0.5: 64898668.0000 - fn0.5: 2798785.0000 - precision0.5: 0.8379 - recall0.5: 0.7662 - tp0.7: 8179901.0000 - fp0.7: 877946.0000 - tn0.7: 65795468.0000 - fn0.7: 3789866.0000 - pr
ecision0.7: 0.9031 - recall0.7: 0.6834 - tp0.9: 6375174.0000 - fp0.9: 261023.0000 - tn0.9: 66412392.0000 - fn0.9: 5594593.0000 - precision0.9: 0.9607 - recall0.9: 0.5326 - accuracy: 0.9418
 - auc: 0.9378 - f1: 0.2642 - val_loss: 1.3899 - val_tp0.1: 3117692.0000 - val_fp0.1: 9912153.0000 - val_tn0.1: 6585793.0000 - val_fn0.1: 45162.0000 - val_precision0.1: 0.2393 - val_recall
0.1: 0.9857 - val_tp0.3: 3079025.0000 - val_fp0.3: 7898193.0000 - val_tn0.3: 8599753.0000 - val_fn0.3: 83829.0000 - val_precision0.3: 0.2805 - val_recall0.3: 0.9735 - val_tp0.5: 3048909.00
00 - val_fp0.5: 6769491.0000 - val_tn0.5: 9728455.0000 - val_fn0.5: 113945.0000 - val_precision0.5: 0.3105 - val_recall0.5: 0.9640 - val_tp0.7: 2994971.0000 - val_fp0.7: 5019595.0000 - val
_tn0.7: 11478351.0000 - val_fn0.7: 167883.0000 - val_precision0.7: 0.3737 - val_recall0.7: 0.9469 - val_tp0.9: 2874370.0000 - val_fp0.9: 2531864.0000 - val_tn0.9: 13966082.0000 - val_fn0.9
: 288484.0000 - val_precision0.9: 0.5317 - val_recall0.9: 0.9088 - val_accuracy: 0.6499 - val_auc: 0.9447 - val_f1: 0.2772
Epoch 5/20
240/240 [==============================] - 124s 517ms/step - loss: 0.4466 - tp0.1: 10915404.0000 - fp0.1: 7994592.0000 - tn0.1: 58678848.0000 - fn0.1: 1054363.0000 - precision0.1: 0.5772 -
 recall0.1: 0.9119 - tp0.3: 10101214.0000 - fp0.3: 3354053.0000 - tn0.3: 63319376.0000 - fn0.3: 1868553.0000 - precision0.3: 0.7507 - recall0.3: 0.8439 - tp0.5: 9363723.0000 - fp0.5: 17904
83.0000 - tn0.5: 64882964.0000 - fn0.5: 2606044.0000 - precision0.5: 0.8395 - recall0.5: 0.7823 - tp0.7: 8326836.0000 - fp0.7: 857418.0000 - tn0.7: 65816000.0000 - fn0.7: 3642931.0000 - pr
ecision0.7: 0.9066 - recall0.7: 0.6957 - tp0.9: 6486870.0000 - fp0.9: 255970.0000 - tn0.9: 66417480.0000 - fn0.9: 5482897.0000 - precision0.9: 0.9620 - recall0.9: 0.5419 - accuracy: 0.9441
 - auc: 0.9425 - f1: 0.2642 - val_loss: 0.9230 - val_tp0.1: 3093810.0000 - val_fp0.1: 6728709.0000 - val_tn0.1: 9769237.0000 - val_fn0.1: 69044.0000 - val_precision0.1: 0.3150 - val_recall
0.1: 0.9782 - val_tp0.3: 2983445.0000 - val_fp0.3: 3338469.0000 - val_tn0.3: 13159477.0000 - val_fn0.3: 179409.0000 - val_precision0.3: 0.4719 - val_recall0.3: 0.9433 - val_tp0.5: 2913456.
0000 - val_fp0.5: 2448590.0000 - val_tn0.5: 14049356.0000 - val_fn0.5: 249398.0000 - val_precision0.5: 0.5433 - val_recall0.5: 0.9211 - val_tp0.7: 2826495.0000 - val_fp0.7: 1728169.0000 -
val_tn0.7: 14769777.0000 - val_fn0.7: 336359.0000 - val_precision0.7: 0.6206 - val_recall0.7: 0.8937 - val_tp0.9: 2684311.0000 - val_fp0.9: 1097331.0000 - val_tn0.9: 15400615.0000 - val_fn
0.9: 478543.0000 - val_precision0.9: 0.7098 - val_recall0.9: 0.8487 - val_accuracy: 0.8628 - val_auc: 0.9474 - val_f1: 0.2772
Epoch 6/20
240/240 [==============================] - 124s 516ms/step - loss: 0.4071 - tp0.1: 11022248.0000 - fp0.1: 7374717.0000 - tn0.1: 59298728.0000 - fn0.1: 947519.0000 - precision0.1: 0.5991 -
recall0.1: 0.9208 - tp0.3: 10241896.0000 - fp0.3: 3036735.0000 - tn0.3: 63636704.0000 - fn0.3: 1727871.0000 - precision0.3: 0.7713 - recall0.3: 0.8556 - tp0.5: 9584629.0000 - fp0.5: 165806
9.0000 - tn0.5: 65015368.0000 - fn0.5: 2385138.0000 - precision0.5: 0.8525 - recall0.5: 0.8007 - tp0.7: 8694782.0000 - fp0.7: 820555.0000 - tn0.7: 65852888.0000 - fn0.7: 3274985.0000 - pre
cision0.7: 0.9138 - recall0.7: 0.7264 - tp0.9: 6966857.0000 - fp0.9: 241359.0000 - tn0.9: 66432068.0000 - fn0.9: 5002910.0000 - precision0.9: 0.9665 - recall0.9: 0.5820 - accuracy: 0.9486
- auc: 0.9486 - f1: 0.2642 - val_loss: 0.4390 - val_tp0.1: 2785859.0000 - val_fp0.1: 688884.0000 - val_tn0.1: 15809062.0000 - val_fn0.1: 376995.0000 - val_precision0.1: 0.8017 - val_recall
0.1: 0.8808 - val_tp0.3: 2458538.0000 - val_fp0.3: 247679.0000 - val_tn0.3: 16250267.0000 - val_fn0.3: 704316.0000 - val_precision0.3: 0.9085 - val_recall0.3: 0.7773 - val_tp0.5: 2155525.0
000 - val_fp0.5: 109573.0000 - val_tn0.5: 16388373.0000 - val_fn0.5: 1007329.0000 - val_precision0.5: 0.9516 - val_recall0.5: 0.6815 - val_tp0.7: 1802828.0000 - val_fp0.7: 45112.0000 - val
_tn0.7: 16452834.0000 - val_fn0.7: 1360026.0000 - val_precision0.7: 0.9756 - val_recall0.7: 0.5700 - val_tp0.9: 1314546.0000 - val_fp0.9: 12049.0000 - val_tn0.9: 16485897.0000 - val_fn0.9:
 1848308.0000 - val_precision0.9: 0.9909 - val_recall0.9: 0.4156 - val_accuracy: 0.9432 - val_auc: 0.9435 - val_f1: 0.2772
Epoch 7/20
240/240 [==============================] - 124s 516ms/step - loss: 0.3853 - tp0.1: 11079386.0000 - fp0.1: 6940988.0000 - tn0.1: 59732460.0000 - fn0.1: 890381.0000 - precision0.1: 0.6148 -
recall0.1: 0.9256 - tp0.3: 10346458.0000 - fp0.3: 3000615.0000 - tn0.3: 63672804.0000 - fn0.3: 1623309.0000 - precision0.3: 0.7752 - recall0.3: 0.8644 - tp0.5: 9690133.0000 - fp0.5: 161636
9.0000 - tn0.5: 65057072.0000 - fn0.5: 2279634.0000 - precision0.5: 0.8570 - recall0.5: 0.8096 - tp0.7: 8843112.0000 - fp0.7: 822086.0000 - tn0.7: 65851372.0000 - fn0.7: 3126655.0000 - pre
cision0.7: 0.9149 - recall0.7: 0.7388 - tp0.9: 7135245.0000 - fp0.9: 238695.0000 - tn0.9: 66434744.0000 - fn0.9: 4834522.0000 - precision0.9: 0.9676 - recall0.9: 0.5961 - accuracy: 0.9505
- auc: 0.9518 - f1: 0.2642 - val_loss: 0.4072 - val_tp0.1: 3071232.0000 - val_fp0.1: 3190431.0000 - val_tn0.1: 13307515.0000 - val_fn0.1: 91622.0000 - val_precision0.1: 0.4905 - val_recall
0.1: 0.9710 - val_tp0.3: 2969302.0000 - val_fp0.3: 1516252.0000 - val_tn0.3: 14981694.0000 - val_fn0.3: 193552.0000 - val_precision0.3: 0.6620 - val_recall0.3: 0.9388 - val_tp0.5: 2813331.
0000 - val_fp0.5: 766580.0000 - val_tn0.5: 15731366.0000 - val_fn0.5: 349523.0000 - val_precision0.5: 0.7859 - val_recall0.5: 0.8895 - val_tp0.7: 2565842.0000 - val_fp0.7: 340498.0000 - va
l_tn0.7: 16157448.0000 - val_fn0.7: 597012.0000 - val_precision0.7: 0.8828 - val_recall0.7: 0.8112 - val_tp0.9: 2014990.0000 - val_fp0.9: 79766.0000 - val_tn0.9: 16418180.0000 - val_fn0.9:
 1147864.0000 - val_precision0.9: 0.9619 - val_recall0.9: 0.6371 - val_accuracy: 0.9432 - val_auc: 0.9709 - val_f1: 0.2772
Epoch 8/20
240/240 [==============================] - 124s 517ms/step - loss: 0.3633 - tp0.1: 11135897.0000 - fp0.1: 6660915.0000 - tn0.1: 60012532.0000 - fn0.1: 833870.0000 - precision0.1: 0.6257 -
recall0.1: 0.9303 - tp0.3: 10437033.0000 - fp0.3: 2736926.0000 - tn0.3: 63936496.0000 - fn0.3: 1532734.0000 - precision0.3: 0.7922 - recall0.3: 0.8719 - tp0.5: 9858968.0000 - fp0.5: 152258
9.0000 - tn0.5: 65150856.0000 - fn0.5: 2110799.0000 - precision0.5: 0.8662 - recall0.5: 0.8237 - tp0.7: 9048624.0000 - fp0.7: 753519.0000 - tn0.7: 65919908.0000 - fn0.7: 2921143.0000 - pre
cision0.7: 0.9231 - recall0.7: 0.7560 - tp0.9: 7412042.0000 - fp0.9: 213551.0000 - tn0.9: 66459872.0000 - fn0.9: 4557725.0000 - precision0.9: 0.9720 - recall0.9: 0.6192 - accuracy: 0.9538
- auc: 0.9558 - f1: 0.2642 - val_loss: 0.4570 - val_tp0.1: 3014123.0000 - val_fp0.1: 2699685.0000 - val_tn0.1: 13798261.0000 - val_fn0.1: 148731.0000 - val_precision0.1: 0.5275 - val_recal
l0.1: 0.9530 - val_tp0.3: 2936029.0000 - val_fp0.3: 1673840.0000 - val_tn0.3: 14824106.0000 - val_fn0.3: 226825.0000 - val_precision0.3: 0.6369 - val_recall0.3: 0.9283 - val_tp0.5: 2861446
.0000 - val_fp0.5: 1230626.0000 - val_tn0.5: 15267320.0000 - val_fn0.5: 301408.0000 - val_precision0.5: 0.6993 - val_recall0.5: 0.9047 - val_tp0.7: 2733900.0000 - val_fp0.7: 781063.0000 -
val_tn0.7: 15716883.0000 - val_fn0.7: 428954.0000 - val_precision0.7: 0.7778 - val_recall0.7: 0.8644 - val_tp0.9: 2482216.0000 - val_fp0.9: 390029.0000 - val_tn0.9: 16107917.0000 - val_fn0
.9: 680638.0000 - val_precision0.9: 0.8642 - val_recall0.9: 0.7848 - val_accuracy: 0.9221 - val_auc: 0.9600 - val_f1: 0.2772
Epoch 9/20
240/240 [==============================] - 124s 518ms/step - loss: 0.3559 - tp0.1: 11134313.0000 - fp0.1: 6236820.0000 - tn0.1: 60436612.0000 - fn0.1: 835454.0000 - precision0.1: 0.6410 -
recall0.1: 0.9302 - tp0.3: 10469952.0000 - fp0.3: 2640391.0000 - tn0.3: 64033040.0000 - fn0.3: 1499815.0000 - precision0.3: 0.7986 - recall0.3: 0.8747 - tp0.5: 9934376.0000 - fp0.5: 152988
9.0000 - tn0.5: 65143560.0000 - fn0.5: 2035391.0000 - precision0.5: 0.8666 - recall0.5: 0.8300 - tp0.7: 9136907.0000 - fp0.7: 753965.0000 - tn0.7: 65919460.0000 - fn0.7: 2832860.0000 - pre
cision0.7: 0.9238 - recall0.7: 0.7633 - tp0.9: 7562687.0000 - fp0.9: 213280.0000 - tn0.9: 66460156.0000 - fn0.9: 4407080.0000 - precision0.9: 0.9726 - recall0.9: 0.6318 - accuracy: 0.9547
- auc: 0.9561 - f1: 0.2642 - val_loss: 0.3791 - val_tp0.1: 2856464.0000 - val_fp0.1: 956306.0000 - val_tn0.1: 15541640.0000 - val_fn0.1: 306390.0000 - val_precision0.1: 0.7492 - val_recall
0.1: 0.9031 - val_tp0.3: 2644444.0000 - val_fp0.3: 392043.0000 - val_tn0.3: 16105903.0000 - val_fn0.3: 518410.0000 - val_precision0.3: 0.8709 - val_recall0.3: 0.8361 - val_tp0.5: 2503008.0
000 - val_fp0.5: 246935.0000 - val_tn0.5: 16251011.0000 - val_fn0.5: 659846.0000 - val_precision0.5: 0.9102 - val_recall0.5: 0.7914 - val_tp0.7: 2293518.0000 - val_fp0.7: 135876.0000 - val
_tn0.7: 16362070.0000 - val_fn0.7: 869336.0000 - val_precision0.7: 0.9441 - val_recall0.7: 0.7251 - val_tp0.9: 1878107.0000 - val_fp0.9: 37476.0000 - val_tn0.9: 16460470.0000 - val_fn0.9:
1284747.0000 - val_precision0.9: 0.9804 - val_recall0.9: 0.5938 - val_accuracy: 0.9539 - val_auc: 0.9458 - val_f1: 0.2772
Epoch 10/20
240/240 [==============================] - 124s 517ms/step - loss: 0.3399 - tp0.1: 11192783.0000 - fp0.1: 6148473.0000 - tn0.1: 60524940.0000 - fn0.1: 776984.0000 - precision0.1: 0.6454 -
recall0.1: 0.9351 - tp0.3: 10562088.0000 - fp0.3: 2647342.0000 - tn0.3: 64026076.0000 - fn0.3: 1407679.0000 - precision0.3: 0.7996 - recall0.3: 0.8824 - tp0.5: 10019446.0000 - fp0.5: 14919
46.0000 - tn0.5: 65181488.0000 - fn0.5: 1950321.0000 - precision0.5: 0.8704 - recall0.5: 0.8371 - tp0.7: 9263641.0000 - fp0.7: 747160.0000 - tn0.7: 65926272.0000 - fn0.7: 2706126.0000 - pr
ecision0.7: 0.9254 - recall0.7: 0.7739 - tp0.9: 7720186.0000 - fp0.9: 209487.0000 - tn0.9: 66463960.0000 - fn0.9: 4249581.0000 - precision0.9: 0.9736 - recall0.9: 0.6450 - accuracy: 0.9562
 - auc: 0.9595 - f1: 0.2642 - val_loss: 0.3522 - val_tp0.1: 3064550.0000 - val_fp0.1: 2487874.0000 - val_tn0.1: 14010072.0000 - val_fn0.1: 98304.0000 - val_precision0.1: 0.5519 - val_recal
l0.1: 0.9689 - val_tp0.3: 2948761.0000 - val_fp0.3: 1064135.0000 - val_tn0.3: 15433811.0000 - val_fn0.3: 214093.0000 - val_precision0.3: 0.7348 - val_recall0.3: 0.9323 - val_tp0.5: 2865468
.0000 - val_fp0.5: 718289.0000 - val_tn0.5: 15779657.0000 - val_fn0.5: 297386.0000 - val_precision0.5: 0.7996 - val_recall0.5: 0.9060 - val_tp0.7: 2741942.0000 - val_fp0.7: 426555.0000 - v
al_tn0.7: 16071391.0000 - val_fn0.7: 420912.0000 - val_precision0.7: 0.8654 - val_recall0.7: 0.8669 - val_tp0.9: 2476605.0000 - val_fp0.9: 171677.0000 - val_tn0.9: 16326269.0000 - val_fn0.
9: 686249.0000 - val_precision0.9: 0.9352 - val_recall0.9: 0.7830 - val_accuracy: 0.9483 - val_auc: 0.9739 - val_f1: 0.2772
Epoch 11/20
240/240 [==============================] - 124s 517ms/step - loss: 0.3283 - tp0.1: 11237515.0000 - fp0.1: 6025317.0000 - tn0.1: 60648124.0000 - fn0.1: 732252.0000 - precision0.1: 0.6510 -
recall0.1: 0.9388 - tp0.3: 10617524.0000 - fp0.3: 2587448.0000 - tn0.3: 64085984.0000 - fn0.3: 1352243.0000 - precision0.3: 0.8041 - recall0.3: 0.8870 - tp0.5: 10090900.0000 - fp0.5: 14806
86.0000 - tn0.5: 65192740.0000 - fn0.5: 1878867.0000 - precision0.5: 0.8720 - recall0.5: 0.8430 - tp0.7: 9347944.0000 - fp0.7: 746924.0000 - tn0.7: 65926508.0000 - fn0.7: 2621823.0000 - pr
ecision0.7: 0.9260 - recall0.7: 0.7810 - tp0.9: 7842670.0000 - fp0.9: 222008.0000 - tn0.9: 66451408.0000 - fn0.9: 4127097.0000 - precision0.9: 0.9725 - recall0.9: 0.6552 - accuracy: 0.9573
 - auc: 0.9614 - f1: 0.2642 - val_loss: 0.3862 - val_tp0.1: 3085105.0000 - val_fp0.1: 3480345.0000 - val_tn0.1: 13017601.0000 - val_fn0.1: 77749.0000 - val_precision0.1: 0.4699 - val_recal
l0.1: 0.9754 - val_tp0.3: 2931914.0000 - val_fp0.3: 986226.0000 - val_tn0.3: 15511720.0000 - val_fn0.3: 230940.0000 - val_precision0.3: 0.7483 - val_recall0.3: 0.9270 - val_tp0.5: 2759869.
0000 - val_fp0.5: 439560.0000 - val_tn0.5: 16058386.0000 - val_fn0.5: 402985.0000 - val_precision0.5: 0.8626 - val_recall0.5: 0.8726 - val_tp0.7: 2487808.0000 - val_fp0.7: 172315.0000 - va
l_tn0.7: 16325631.0000 - val_fn0.7: 675046.0000 - val_precision0.7: 0.9352 - val_recall0.7: 0.7866 - val_tp0.9: 1923553.0000 - val_fp0.9: 34623.0000 - val_tn0.9: 16463323.0000 - val_fn0.9:
 1239301.0000 - val_precision0.9: 0.9823 - val_recall0.9: 0.6082 - val_accuracy: 0.9571 - val_auc: 0.9755 - val_f1: 0.2772
Epoch 12/20
240/240 [==============================] - 124s 517ms/step - loss: 0.3149 - tp0.1: 11244552.0000 - fp0.1: 5636478.0000 - tn0.1: 61036940.0000 - fn0.1: 725215.0000 - precision0.1: 0.6661 -
recall0.1: 0.9394 - tp0.3: 10645433.0000 - fp0.3: 2370845.0000 - tn0.3: 64302560.0000 - fn0.3: 1324334.0000 - precision0.3: 0.8179 - recall0.3: 0.8894 - tp0.5: 10139252.0000 - fp0.5: 13412
79.0000 - tn0.5: 65332156.0000 - fn0.5: 1830515.0000 - precision0.5: 0.8832 - recall0.5: 0.8471 - tp0.7: 9434975.0000 - fp0.7: 668796.0000 - tn0.7: 66004652.0000 - fn0.7: 2534792.0000 - pr
ecision0.7: 0.9338 - recall0.7: 0.7882 - tp0.9: 8010285.0000 - fp0.9: 191192.0000 - tn0.9: 66482252.0000 - fn0.9: 3959482.0000 - precision0.9: 0.9767 - recall0.9: 0.6692 - accuracy: 0.9597
 - auc: 0.9629 - f1: 0.2642 - val_loss: 0.3954 - val_tp0.1: 3008877.0000 - val_fp0.1: 2662719.0000 - val_tn0.1: 13835227.0000 - val_fn0.1: 153977.0000 - val_precision0.1: 0.5305 - val_reca
ll0.1: 0.9513 - val_tp0.3: 2890670.0000 - val_fp0.3: 1167312.0000 - val_tn0.3: 15330634.0000 - val_fn0.3: 272184.0000 - val_precision0.3: 0.7123 - val_recall0.3: 0.9139 - val_tp0.5: 279631
0.0000 - val_fp0.5: 773358.0000 - val_tn0.5: 15724588.0000 - val_fn0.5: 366544.0000 - val_precision0.5: 0.7834 - val_recall0.5: 0.8841 - val_tp0.7: 2635367.0000 - val_fp0.7: 412051.0000 -
val_tn0.7: 16085895.0000 - val_fn0.7: 527487.0000 - val_precision0.7: 0.8648 - val_recall0.7: 0.8332 - val_tp0.9: 2302238.0000 - val_fp0.9: 140209.0000 - val_tn0.9: 16357737.0000 - val_fn0
.9: 860616.0000 - val_precision0.9: 0.9426 - val_recall0.9: 0.7279 - val_accuracy: 0.9420 - val_auc: 0.9629 - val_f1: 0.2772
Epoch 13/20
240/240 [==============================] - 124s 516ms/step - loss: 0.3001 - tp0.1: 11285735.0000 - fp0.1: 5401576.0000 - tn0.1: 61271836.0000 - fn0.1: 684032.0000 - precision0.1: 0.6763 -
recall0.1: 0.9429 - tp0.3: 10728442.0000 - fp0.3: 2362125.0000 - tn0.3: 64311320.0000 - fn0.3: 1241325.0000 - precision0.3: 0.8196 - recall0.3: 0.8963 - tp0.5: 10253499.0000 - fp0.5: 13693
34.0000 - tn0.5: 65304092.0000 - fn0.5: 1716268.0000 - precision0.5: 0.8822 - recall0.5: 0.8566 - tp0.7: 9558014.0000 - fp0.7: 688333.0000 - tn0.7: 65985076.0000 - fn0.7: 2411753.0000 - pr
ecision0.7: 0.9328 - recall0.7: 0.7985 - tp0.9: 8134971.0000 - fp0.9: 193269.0000 - tn0.9: 66480164.0000 - fn0.9: 3834796.0000 - precision0.9: 0.9768 - recall0.9: 0.6796 - accuracy: 0.9608
 - auc: 0.9650 - f1: 0.2642 - val_loss: 0.3591 - val_tp0.1: 2894724.0000 - val_fp0.1: 841367.0000 - val_tn0.1: 15656579.0000 - val_fn0.1: 268130.0000 - val_precision0.1: 0.7748 - val_recal
l0.1: 0.9152 - val_tp0.3: 2569647.0000 - val_fp0.3: 243687.0000 - val_tn0.3: 16254259.0000 - val_fn0.3: 593207.0000 - val_precision0.3: 0.9134 - val_recall0.3: 0.8124 - val_tp0.5: 2323874.
0000 - val_fp0.5: 112189.0000 - val_tn0.5: 16385757.0000 - val_fn0.5: 838980.0000 - val_precision0.5: 0.9539 - val_recall0.5: 0.7347 - val_tp0.7: 1996399.0000 - val_fp0.7: 37554.0000 - val
_tn0.7: 16460392.0000 - val_fn0.7: 1166455.0000 - val_precision0.7: 0.9815 - val_recall0.7: 0.6312 - val_tp0.9: 1499767.0000 - val_fp0.9: 5568.0000 - val_tn0.9: 16492378.0000 - val_fn0.9:
1663087.0000 - val_precision0.9: 0.9963 - val_recall0.9: 0.4742 - val_accuracy: 0.9516 - val_auc: 0.9540 - val_f1: 0.2772
Epoch 14/20
240/240 [==============================] - 124s 516ms/step - loss: 0.2927 - tp0.1: 11303319.0000 - fp0.1: 5310555.0000 - tn0.1: 61362872.0000 - fn0.1: 666448.0000 - precision0.1: 0.6804 -
recall0.1: 0.9443 - tp0.3: 10769572.0000 - fp0.3: 2338920.0000 - tn0.3: 64334512.0000 - fn0.3: 1200195.0000 - precision0.3: 0.8216 - recall0.3: 0.8997 - tp0.5: 10293662.0000 - fp0.5: 13284
81.0000 - tn0.5: 65344964.0000 - fn0.5: 1676105.0000 - precision0.5: 0.8857 - recall0.5: 0.8600 - tp0.7: 9608679.0000 - fp0.7: 652337.0000 - tn0.7: 66021108.0000 - fn0.7: 2361088.0000 - pr
ecision0.7: 0.9364 - recall0.7: 0.8027 - tp0.9: 8281304.0000 - fp0.9: 189171.0000 - tn0.9: 66484272.0000 - fn0.9: 3688463.0000 - precision0.9: 0.9777 - recall0.9: 0.6919 - accuracy: 0.9618
 - auc: 0.9660 - f1: 0.2642 - val_loss: 0.3489 - val_tp0.1: 3071912.0000 - val_fp0.1: 2477377.0000 - val_tn0.1: 14020569.0000 - val_fn0.1: 90942.0000 - val_precision0.1: 0.5536 - val_recal
l0.1: 0.9712 - val_tp0.3: 2983731.0000 - val_fp0.3: 1182747.0000 - val_tn0.3: 15315199.0000 - val_fn0.3: 179123.0000 - val_precision0.3: 0.7161 - val_recall0.3: 0.9434 - val_tp0.5: 2894503
.0000 - val_fp0.5: 758141.0000 - val_tn0.5: 15739805.0000 - val_fn0.5: 268351.0000 - val_precision0.5: 0.7924 - val_recall0.5: 0.9152 - val_tp0.7: 2765944.0000 - val_fp0.7: 443418.0000 - v
al_tn0.7: 16054528.0000 - val_fn0.7: 396910.0000 - val_precision0.7: 0.8618 - val_recall0.7: 0.8745 - val_tp0.9: 2524618.0000 - val_fp0.9: 206040.0000 - val_tn0.9: 16291906.0000 - val_fn0.
9: 638236.0000 - val_precision0.9: 0.9245 - val_recall0.9: 0.7982 - val_accuracy: 0.9478 - val_auc: 0.9752 - val_f1: 0.2772
Epoch 15/20
240/240 [==============================] - 124s 516ms/step - loss: 0.2859 - tp0.1: 11302324.0000 - fp0.1: 4990158.0000 - tn0.1: 61683280.0000 - fn0.1: 667443.0000 - precision0.1: 0.6937 -
recall0.1: 0.9442 - tp0.3: 10801269.0000 - fp0.3: 2296897.0000 - tn0.3: 64376528.0000 - fn0.3: 1168498.0000 - precision0.3: 0.8246 - recall0.3: 0.9024 - tp0.5: 10322016.0000 - fp0.5: 12928
01.0000 - tn0.5: 65380632.0000 - fn0.5: 1647751.0000 - precision0.5: 0.8887 - recall0.5: 0.8623 - tp0.7: 9638120.0000 - fp0.7: 622814.0000 - tn0.7: 66050612.0000 - fn0.7: 2331647.0000 - pr
ecision0.7: 0.9393 - recall0.7: 0.8052 - tp0.9: 8333108.0000 - fp0.9: 176077.0000 - tn0.9: 66497368.0000 - fn0.9: 3636659.0000 - precision0.9: 0.9793 - recall0.9: 0.6962 - accuracy: 0.9626
 - auc: 0.9664 - f1: 0.2642 - val_loss: 0.2609 - val_tp0.1: 3046775.0000 - val_fp0.1: 1468094.0000 - val_tn0.1: 15029852.0000 - val_fn0.1: 116079.0000 - val_precision0.1: 0.6748 - val_reca
ll0.1: 0.9633 - val_tp0.3: 2926133.0000 - val_fp0.3: 616330.0000 - val_tn0.3: 15881616.0000 - val_fn0.3: 236721.0000 - val_precision0.3: 0.8260 - val_recall0.3: 0.9252 - val_tp0.5: 2805534
.0000 - val_fp0.5: 344220.0000 - val_tn0.5: 16153726.0000 - val_fn0.5: 357320.0000 - val_precision0.5: 0.8907 - val_recall0.5: 0.8870 - val_tp0.7: 2613829.0000 - val_fp0.7: 161631.0000 - v
al_tn0.7: 16336315.0000 - val_fn0.7: 549025.0000 - val_precision0.7: 0.9418 - val_recall0.7: 0.8264 - val_tp0.9: 2262420.0000 - val_fp0.9: 50786.0000 - val_tn0.9: 16447160.0000 - val_fn0.9
: 900434.0000 - val_precision0.9: 0.9780 - val_recall0.9: 0.7153 - val_accuracy: 0.9643 - val_auc: 0.9756 - val_f1: 0.2772
Epoch 16/20
240/240 [==============================] - 124s 518ms/step - loss: 0.2760 - tp0.1: 11343439.0000 - fp0.1: 5034724.0000 - tn0.1: 61638692.0000 - fn0.1: 626328.0000 - precision0.1: 0.6926 -
recall0.1: 0.9477 - tp0.3: 10839346.0000 - fp0.3: 2204383.0000 - tn0.3: 64469040.0000 - fn0.3: 1130421.0000 - precision0.3: 0.8310 - recall0.3: 0.9056 - tp0.5: 10384981.0000 - fp0.5: 12476
08.0000 - tn0.5: 65425836.0000 - fn0.5: 1584786.0000 - precision0.5: 0.8927 - recall0.5: 0.8676 - tp0.7: 9712329.0000 - fp0.7: 596551.0000 - tn0.7: 66076892.0000 - fn0.7: 2257438.0000 - pr
ecision0.7: 0.9421 - recall0.7: 0.8114 - tp0.9: 8432551.0000 - fp0.9: 162062.0000 - tn0.9: 66511360.0000 - fn0.9: 3537216.0000 - precision0.9: 0.9811 - recall0.9: 0.7045 - accuracy: 0.9640
 - auc: 0.9683 - f1: 0.2642 - val_loss: 0.2563 - val_tp0.1: 3029508.0000 - val_fp0.1: 1192309.0000 - val_tn0.1: 15305637.0000 - val_fn0.1: 133346.0000 - val_precision0.1: 0.7176 - val_reca
ll0.1: 0.9578 - val_tp0.3: 2908219.0000 - val_fp0.3: 564874.0000 - val_tn0.3: 15933072.0000 - val_fn0.3: 254635.0000 - val_precision0.3: 0.8374 - val_recall0.3: 0.9195 - val_tp0.5: 2799190
.0000 - val_fp0.5: 352643.0000 - val_tn0.5: 16145303.0000 - val_fn0.5: 363664.0000 - val_precision0.5: 0.8881 - val_recall0.5: 0.8850 - val_tp0.7: 2636887.0000 - val_fp0.7: 194123.0000 - v
al_tn0.7: 16303823.0000 - val_fn0.7: 525967.0000 - val_precision0.7: 0.9314 - val_recall0.7: 0.8337 - val_tp0.9: 2320546.0000 - val_fp0.9: 72127.0000 - val_tn0.9: 16425819.0000 - val_fn0.9
: 842308.0000 - val_precision0.9: 0.9699 - val_recall0.9: 0.7337 - val_accuracy: 0.9636 - val_auc: 0.9731 - val_f1: 0.2772
Epoch 17/20
240/240 [==============================] - 124s 518ms/step - loss: 0.2703 - tp0.1: 11357057.0000 - fp0.1: 4893464.0000 - tn0.1: 61779992.0000 - fn0.1: 612710.0000 - precision0.1: 0.6989 -
recall0.1: 0.9488 - tp0.3: 10875567.0000 - fp0.3: 2221183.0000 - tn0.3: 64452244.0000 - fn0.3: 1094200.0000 - precision0.3: 0.8304 - recall0.3: 0.9086 - tp0.5: 10410193.0000 - fp0.5: 12391
75.0000 - tn0.5: 65434264.0000 - fn0.5: 1559574.0000 - precision0.5: 0.8936 - recall0.5: 0.8697 - tp0.7: 9743863.0000 - fp0.7: 592187.0000 - tn0.7: 66081244.0000 - fn0.7: 2225904.0000 - pr
ecision0.7: 0.9427 - recall0.7: 0.8140 - tp0.9: 8521739.0000 - fp0.9: 167850.0000 - tn0.9: 66505612.0000 - fn0.9: 3448028.0000 - precision0.9: 0.9807 - recall0.9: 0.7119 - accuracy: 0.9644
 - auc: 0.9690 - f1: 0.2642 - val_loss: 0.2701 - val_tp0.1: 3061862.0000 - val_fp0.1: 1755743.0000 - val_tn0.1: 14742203.0000 - val_fn0.1: 100992.0000 - val_precision0.1: 0.6356 - val_reca
ll0.1: 0.9681 - val_tp0.3: 2945795.0000 - val_fp0.3: 710061.0000 - val_tn0.3: 15787885.0000 - val_fn0.3: 217059.0000 - val_precision0.3: 0.8058 - val_recall0.3: 0.9314 - val_tp0.5: 2833502
.0000 - val_fp0.5: 394281.0000 - val_tn0.5: 16103665.0000 - val_fn0.5: 329352.0000 - val_precision0.5: 0.8778 - val_recall0.5: 0.8959 - val_tp0.7: 2650030.0000 - val_fp0.7: 181493.0000 - v
al_tn0.7: 16316453.0000 - val_fn0.7: 512824.0000 - val_precision0.7: 0.9359 - val_recall0.7: 0.8379 - val_tp0.9: 2293107.0000 - val_fp0.9: 49911.0000 - val_tn0.9: 16448035.0000 - val_fn0.9
: 869747.0000 - val_precision0.9: 0.9787 - val_recall0.9: 0.7250 - val_accuracy: 0.9632 - val_auc: 0.9773 - val_f1: 0.2772
Epoch 18/20
240/240 [==============================] - 124s 518ms/step - loss: 0.2628 - tp0.1: 11372952.0000 - fp0.1: 4780985.0000 - tn0.1: 61892464.0000 - fn0.1: 596815.0000 - precision0.1: 0.7040 -
recall0.1: 0.9501 - tp0.3: 10899958.0000 - fp0.3: 2139815.0000 - tn0.3: 64533632.0000 - fn0.3: 1069809.0000 - precision0.3: 0.8359 - recall0.3: 0.9106 - tp0.5: 10471625.0000 - fp0.5: 12249
84.0000 - tn0.5: 65448432.0000 - fn0.5: 1498142.0000 - precision0.5: 0.8953 - recall0.5: 0.8748 - tp0.7: 9815694.0000 - fp0.7: 581437.0000 - tn0.7: 66091992.0000 - fn0.7: 2154073.0000 - pr
ecision0.7: 0.9441 - recall0.7: 0.8200 - tp0.9: 8590979.0000 - fp0.9: 158690.0000 - tn0.9: 66514760.0000 - fn0.9: 3378788.0000 - precision0.9: 0.9819 - recall0.9: 0.7177 - accuracy: 0.9654
 - auc: 0.9700 - f1: 0.2642 - val_loss: 0.2858 - val_tp0.1: 3079296.0000 - val_fp0.1: 2141364.0000 - val_tn0.1: 14356582.0000 - val_fn0.1: 83558.0000 - val_precision0.1: 0.5898 - val_recal
l0.1: 0.9736 - val_tp0.3: 2982065.0000 - val_fp0.3: 935179.0000 - val_tn0.3: 15562767.0000 - val_fn0.3: 180789.0000 - val_precision0.3: 0.7613 - val_recall0.3: 0.9428 - val_tp0.5: 2881468.
0000 - val_fp0.5: 536262.0000 - val_tn0.5: 15961684.0000 - val_fn0.5: 281386.0000 - val_precision0.5: 0.8431 - val_recall0.5: 0.9110 - val_tp0.7: 2708837.0000 - val_fp0.7: 237110.0000 - va
l_tn0.7: 16260836.0000 - val_fn0.7: 454017.0000 - val_precision0.7: 0.9195 - val_recall0.7: 0.8565 - val_tp0.9: 2296651.0000 - val_fp0.9: 53102.0000 - val_tn0.9: 16444844.0000 - val_fn0.9:
 866203.0000 - val_precision0.9: 0.9774 - val_recall0.9: 0.7261 - val_accuracy: 0.9584 - val_auc: 0.9793 - val_f1: 0.2772
Epoch 19/20
240/240 [==============================] - 124s 516ms/step - loss: 0.2557 - tp0.1: 11390165.0000 - fp0.1: 4618150.0000 - tn0.1: 62055280.0000 - fn0.1: 579602.0000 - precision0.1: 0.7115 -
recall0.1: 0.9516 - tp0.3: 10925273.0000 - fp0.3: 2091134.0000 - tn0.3: 64582276.0000 - fn0.3: 1044494.0000 - precision0.3: 0.8393 - recall0.3: 0.9127 - tp0.5: 10488558.0000 - fp0.5: 11931
41.0000 - tn0.5: 65480284.0000 - fn0.5: 1481209.0000 - precision0.5: 0.8979 - recall0.5: 0.8763 - tp0.7: 9830525.0000 - fp0.7: 561047.0000 - tn0.7: 66112360.0000 - fn0.7: 2139242.0000 - pr
ecision0.7: 0.9460 - recall0.7: 0.8213 - tp0.9: 8625871.0000 - fp0.9: 152724.0000 - tn0.9: 66520724.0000 - fn0.9: 3343896.0000 - precision0.9: 0.9826 - recall0.9: 0.7206 - accuracy: 0.9660
 - auc: 0.9711 - f1: 0.2642 - val_loss: 0.2528 - val_tp0.1: 3043668.0000 - val_fp0.1: 1335624.0000 - val_tn0.1: 15162322.0000 - val_fn0.1: 119186.0000 - val_precision0.1: 0.6950 - val_reca
ll0.1: 0.9623 - val_tp0.3: 2935979.0000 - val_fp0.3: 619716.0000 - val_tn0.3: 15878230.0000 - val_fn0.3: 226875.0000 - val_precision0.3: 0.8257 - val_recall0.3: 0.9283 - val_tp0.5: 2834409
.0000 - val_fp0.5: 367871.0000 - val_tn0.5: 16130075.0000 - val_fn0.5: 328445.0000 - val_precision0.5: 0.8851 - val_recall0.5: 0.8962 - val_tp0.7: 2670269.0000 - val_fp0.7: 182097.0000 - v
al_tn0.7: 16315849.0000 - val_fn0.7: 492585.0000 - val_precision0.7: 0.9362 - val_recall0.7: 0.8443 - val_tp0.9: 2324993.0000 - val_fp0.9: 54375.0000 - val_tn0.9: 16443571.0000 - val_fn0.9
: 837861.0000 - val_precision0.9: 0.9771 - val_recall0.9: 0.7351 - val_accuracy: 0.9646 - val_auc: 0.9758 - val_f1: 0.2772
Epoch 20/20
240/240 [==============================] - 124s 518ms/step - loss: 0.2510 - tp0.1: 11400658.0000 - fp0.1: 4532014.0000 - tn0.1: 62141432.0000 - fn0.1: 569109.0000 - precision0.1: 0.7156 -
recall0.1: 0.9525 - tp0.3: 10961511.0000 - fp0.3: 2108269.0000 - tn0.3: 64565152.0000 - fn0.3: 1008256.0000 - precision0.3: 0.8387 - recall0.3: 0.9158 - tp0.5: 10542090.0000 - fp0.5: 12141
91.0000 - tn0.5: 65459224.0000 - fn0.5: 1427677.0000 - precision0.5: 0.8967 - recall0.5: 0.8807 - tp0.7: 9900824.0000 - fp0.7: 577656.0000 - tn0.7: 66095784.0000 - fn0.7: 2068943.0000 - pr
ecision0.7: 0.9449 - recall0.7: 0.8272 - tp0.9: 8716202.0000 - fp0.9: 157381.0000 - tn0.9: 66516056.0000 - fn0.9: 3253565.0000 - precision0.9: 0.9823 - recall0.9: 0.7282 - accuracy: 0.9664
 - auc: 0.9716 - f1: 0.2642 - val_loss: 0.2602 - val_tp0.1: 3022436.0000 - val_fp0.1: 1215092.0000 - val_tn0.1: 15282854.0000 - val_fn0.1: 140418.0000 - val_precision0.1: 0.7133 - val_reca
ll0.1: 0.9556 - val_tp0.3: 2889313.0000 - val_fp0.3: 511507.0000 - val_tn0.3: 15986439.0000 - val_fn0.3: 273541.0000 - val_precision0.3: 0.8496 - val_recall0.3: 0.9135 - val_tp0.5: 2761628
.0000 - val_fp0.5: 283172.0000 - val_tn0.5: 16214774.0000 - val_fn0.5: 401226.0000 - val_precision0.5: 0.9070 - val_recall0.5: 0.8731 - val_tp0.7: 2568361.0000 - val_fp0.7: 130133.0000 - v
al_tn0.7: 16367813.0000 - val_fn0.7: 594493.0000 - val_precision0.7: 0.9518 - val_recall0.7: 0.8120 - val_tp0.9: 2202966.0000 - val_fp0.9: 36402.0000 - val_tn0.9: 16461544.0000 - val_fn0.9
: 959888.0000 - val_precision0.9: 0.9837 - val_recall0.9: 0.6965 - val_accuracy: 0.9652 - val_auc: 0.9728 - val_f1: 0.2772
--- Running training session 63/140
{'hp_epochs': 20, 'hp_batch_size': 14, 'hp_scaler': 'quant_g', 'hp_n_levels': 7, 'hp_first_filters': 16, 'hp_pool_size': 2, 'hp_input_size': 4096, 'hp_lr_start': 0.027144629354532844, 'hp_
lr_power': 5.0}
--- repeat #: 1
input - shape:   (None, 4096, 1)
output - shape:  (None, 4096, 1)
Epoch 1/20
2021-08-08 13:46:55.765737: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:177] Filling up shuffle buffer (this may take a while): 3624 of 4800
2021-08-08 13:46:58.998412: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:230] Shuffle buffer filled.
342/342 [==============================] - 60s 96ms/step - loss: 0.9388 - tp0.1: 2551056.0000 - fp0.1: 5602226.0000 - tn0.1: 11036673.0000 - fn0.1: 421693.0000 - precision0.1: 0.3129 - rec
all0.1: 0.8581 - tp0.3: 1927079.0000 - fp0.3: 2492968.0000 - tn0.3: 14145931.0000 - fn0.3: 1045670.0000 - precision0.3: 0.4360 - recall0.3: 0.6482 - tp0.5: 1246353.0000 - fp0.5: 839125.000
0 - tn0.5: 15799774.0000 - fn0.5: 1726396.0000 - precision0.5: 0.5976 - recall0.5: 0.4193 - tp0.7: 687790.0000 - fp0.7: 234530.0000 - tn0.7: 16404369.0000 - fn0.7: 2284959.0000 - precision
0.7: 0.7457 - recall0.7: 0.2314 - tp0.9: 115373.0000 - fp0.9: 19205.0000 - tn0.9: 16619694.0000 - fn0.9: 2857376.0000 - precision0.9: 0.8573 - recall0.9: 0.0388 - accuracy: 0.8692 - auc: 0
.8319 - f1: 0.2633 - val_loss: 1.2163 - val_tp0.1: 649575.0000 - val_fp0.1: 1129986.0000 - val_tn0.1: 2947304.0000 - val_fn0.1: 147375.0000 - val_precision0.1: 0.3650 - val_recall0.1: 0.81
51 - val_tp0.3: 600297.0000 - val_fp0.3: 897583.0000 - val_tn0.3: 3179707.0000 - val_fn0.3: 196653.0000 - val_precision0.3: 0.4008 - val_recall0.3: 0.7532 - val_tp0.5: 562997.0000 - val_fp
0.5: 731818.0000 - val_tn0.5: 3345472.0000 - val_fn0.5: 233953.0000 - val_precision0.5: 0.4348 - val_recall0.5: 0.7064 - val_tp0.7: 521584.0000 - val_fp0.7: 583432.0000 - val_tn0.7: 349385
8.0000 - val_fn0.7: 275366.0000 - val_precision0.7: 0.4720 - val_recall0.7: 0.6545 - val_tp0.9: 432196.0000 - val_fp0.9: 357157.0000 - val_tn0.9: 3720133.0000 - val_fn0.9: 364754.0000 - va
l_precision0.9: 0.5475 - val_recall0.9: 0.5423 - val_accuracy: 0.8019 - val_auc: 0.8254 - val_f1: 0.2811
Epoch 2/20
342/342 [==============================] - 27s 79ms/step - loss: 0.8637 - tp0.1: 2563159.0000 - fp0.1: 5123938.0000 - tn0.1: 11520950.0000 - fn0.1: 403601.0000 - precision0.1: 0.3334 - rec
all0.1: 0.8640 - tp0.3: 2082014.0000 - fp0.3: 2404817.0000 - tn0.3: 14240071.0000 - fn0.3: 884746.0000 - precision0.3: 0.4640 - recall0.3: 0.7018 - tp0.5: 1378923.0000 - fp0.5: 865967.0000
 - tn0.5: 15778921.0000 - fn0.5: 1587837.0000 - precision0.5: 0.6142 - recall0.5: 0.4648 - tp0.7: 747621.0000 - fp0.7: 249304.0000 - tn0.7: 16395584.0000 - fn0.7: 2219139.0000 - precision0
.7: 0.7499 - recall0.7: 0.2520 - tp0.9: 156922.0000 - fp0.9: 25309.0000 - tn0.9: 16619579.0000 - fn0.9: 2809838.0000 - precision0.9: 0.8611 - recall0.9: 0.0529 - accuracy: 0.8749 - auc: 0.
8499 - f1: 0.2628 - val_loss: 6.7415 - val_tp0.1: 794760.0000 - val_fp0.1: 3364179.0000 - val_tn0.1: 712620.0000 - val_fn0.1: 2681.0000 - val_precision0.1: 0.1911 - val_recall0.1: 0.9966 -
 val_tp0.3: 790967.0000 - val_fp0.3: 2758328.0000 - val_tn0.3: 1318471.0000 - val_fn0.3: 6474.0000 - val_precision0.3: 0.2229 - val_recall0.3: 0.9919 - val_tp0.5: 786264.0000 - val_fp0.5:
2520461.0000 - val_tn0.5: 1556338.0000 - val_fn0.5: 11177.0000 - val_precision0.5: 0.2378 - val_recall0.5: 0.9860 - val_tp0.7: 779634.0000 - val_fp0.7: 2343349.0000 - val_tn0.7: 1733450.00
00 - val_fn0.7: 17807.0000 - val_precision0.7: 0.2496 - val_recall0.7: 0.9777 - val_tp0.9: 763809.0000 - val_fp0.9: 2124191.0000 - val_tn0.9: 1952608.0000 - val_fn0.9: 33632.0000 - val_pre
cision0.9: 0.2645 - val_recall0.9: 0.9578 - val_accuracy: 0.4806 - val_auc: 0.7467 - val_f1: 0.2812
Epoch 3/20
342/342 [==============================] - 30s 88ms/step - loss: 0.7536 - tp0.1: 2653716.0000 - fp0.1: 4711992.0000 - tn0.1: 11929767.0000 - fn0.1: 316173.0000 - precision0.1: 0.3603 - rec
all0.1: 0.8935 - tp0.3: 2203201.0000 - fp0.3: 2044395.0000 - tn0.3: 14597364.0000 - fn0.3: 766688.0000 - precision0.3: 0.5187 - recall0.3: 0.7418 - tp0.5: 1692995.0000 - fp0.5: 943961.0000
 - tn0.5: 15697798.0000 - fn0.5: 1276894.0000 - precision0.5: 0.6420 - recall0.5: 0.5701 - tp0.7: 1039318.0000 - fp0.7: 338210.0000 - tn0.7: 16303549.0000 - fn0.7: 1930571.0000 - precision
0.7: 0.7545 - recall0.7: 0.3500 - tp0.9: 234516.0000 - fp0.9: 36668.0000 - tn0.9: 16605091.0000 - fn0.9: 2735373.0000 - precision0.9: 0.8648 - recall0.9: 0.0790 - accuracy: 0.8868 - auc: 0
.8820 - f1: 0.2630 - val_loss: 1.5030 - val_tp0.1: 736916.0000 - val_fp0.1: 1516908.0000 - val_tn0.1: 2557738.0000 - val_fn0.1: 62678.0000 - val_precision0.1: 0.3270 - val_recall0.1: 0.921
6 - val_tp0.3: 720401.0000 - val_fp0.3: 1289822.0000 - val_tn0.3: 2784824.0000 - val_fn0.3: 79193.0000 - val_precision0.3: 0.3584 - val_recall0.3: 0.9010 - val_tp0.5: 705808.0000 - val_fp0
.5: 1149637.0000 - val_tn0.5: 2925009.0000 - val_fn0.5: 93786.0000 - val_precision0.5: 0.3804 - val_recall0.5: 0.8827 - val_tp0.7: 685398.0000 - val_fp0.7: 986538.0000 - val_tn0.7: 3088108
.0000 - val_fn0.7: 114196.0000 - val_precision0.7: 0.4099 - val_recall0.7: 0.8572 - val_tp0.9: 644148.0000 - val_fp0.9: 756371.0000 - val_tn0.9: 3318275.0000 - val_fn0.9: 155446.0000 - val
_precision0.9: 0.4599 - val_recall0.9: 0.8056 - val_accuracy: 0.7449 - val_auc: 0.8651 - val_f1: 0.2819
Epoch 4/20
342/342 [==============================] - 30s 88ms/step - loss: 0.6853 - tp0.1: 2703897.0000 - fp0.1: 4342508.0000 - tn0.1: 12288832.0000 - fn0.1: 276411.0000 - precision0.1: 0.3837 - rec
all0.1: 0.9073 - tp0.3: 2293716.0000 - fp0.3: 1900715.0000 - tn0.3: 14730625.0000 - fn0.3: 686592.0000 - precision0.3: 0.5468 - recall0.3: 0.7696 - tp0.5: 1860758.0000 - fp0.5: 965942.0000
 - tn0.5: 15665398.0000 - fn0.5: 1119550.0000 - precision0.5: 0.6583 - recall0.5: 0.6244 - tp0.7: 1232630.0000 - fp0.7: 369641.0000 - tn0.7: 16261699.0000 - fn0.7: 1747678.0000 - precision
0.7: 0.7693 - recall0.7: 0.4136 - tp0.9: 331288.0000 - fp0.9: 50757.0000 - tn0.9: 16580583.0000 - fn0.9: 2649020.0000 - precision0.9: 0.8671 - recall0.9: 0.1112 - accuracy: 0.8937 - auc: 0
.8982 - f1: 0.2638 - val_loss: 0.9547 - val_tp0.1: 735656.0000 - val_fp0.1: 1474945.0000 - val_tn0.1: 2608195.0000 - val_fn0.1: 55444.0000 - val_precision0.1: 0.3328 - val_recall0.1: 0.929
9 - val_tp0.3: 692992.0000 - val_fp0.3: 1096764.0000 - val_tn0.3: 2986376.0000 - val_fn0.3: 98108.0000 - val_precision0.3: 0.3872 - val_recall0.3: 0.8760 - val_tp0.5: 651741.0000 - val_fp0
.5: 858021.0000 - val_tn0.5: 3225119.0000 - val_fn0.5: 139359.0000 - val_precision0.5: 0.4317 - val_recall0.5: 0.8238 - val_tp0.7: 578859.0000 - val_fp0.7: 602934.0000 - val_tn0.7: 3480206
.0000 - val_fn0.7: 212241.0000 - val_precision0.7: 0.4898 - val_recall0.7: 0.7317 - val_tp0.9: 421648.0000 - val_fp0.9: 292420.0000 - val_tn0.9: 3790720.0000 - val_fn0.9: 369452.0000 - val
_precision0.9: 0.5905 - val_recall0.9: 0.5330 - val_accuracy: 0.7954 - val_auc: 0.8742 - val_f1: 0.2793
Epoch 5/20
342/342 [==============================] - 30s 89ms/step - loss: 0.6168 - tp0.1: 2714795.0000 - fp0.1: 3825827.0000 - tn0.1: 12828162.0000 - fn0.1: 242864.0000 - precision0.1: 0.4151 - rec
all0.1: 0.9179 - tp0.3: 2360594.0000 - fp0.3: 1759953.0000 - tn0.3: 14894036.0000 - fn0.3: 597065.0000 - precision0.3: 0.5729 - recall0.3: 0.7981 - tp0.5: 1993009.0000 - fp0.5: 943846.0000
 - tn0.5: 15710143.0000 - fn0.5: 964650.0000 - precision0.5: 0.6786 - recall0.5: 0.6738 - tp0.7: 1426514.0000 - fp0.7: 393147.0000 - tn0.7: 16260842.0000 - fn0.7: 1531145.0000 - precision0
.7: 0.7839 - recall0.7: 0.4823 - tp0.9: 433784.0000 - fp0.9: 59911.0000 - tn0.9: 16594078.0000 - fn0.9: 2523875.0000 - precision0.9: 0.8786 - recall0.9: 0.1467 - accuracy: 0.9027 - auc: 0.
9117 - f1: 0.2621 - val_loss: 0.6248 - val_tp0.1: 740961.0000 - val_fp0.1: 952224.0000 - val_tn0.1: 3122967.0000 - val_fn0.1: 58088.0000 - val_precision0.1: 0.4376 - val_recall0.1: 0.9273
- val_tp0.3: 664317.0000 - val_fp0.3: 488340.0000 - val_tn0.3: 3586851.0000 - val_fn0.3: 134732.0000 - val_precision0.3: 0.5763 - val_recall0.3: 0.8314 - val_tp0.5: 563011.0000 - val_fp0.5
: 245735.0000 - val_tn0.5: 3829456.0000 - val_fn0.5: 236038.0000 - val_precision0.5: 0.6962 - val_recall0.5: 0.7046 - val_tp0.7: 395949.0000 - val_fp0.7: 88632.0000 - val_tn0.7: 3986559.00
00 - val_fn0.7: 403100.0000 - val_precision0.7: 0.8171 - val_recall0.7: 0.4955 - val_tp0.9: 112039.0000 - val_fp0.9: 5626.0000 - val_tn0.9: 4069565.0000 - val_fn0.9: 687010.0000 - val_prec
ision0.9: 0.9522 - val_recall0.9: 0.1402 - val_accuracy: 0.9012 - val_auc: 0.9231 - val_f1: 0.2817
Epoch 6/20
342/342 [==============================] - 29s 84ms/step - loss: 0.5810 - tp0.1: 2723990.0000 - fp0.1: 3427161.0000 - tn0.1: 13216327.0000 - fn0.1: 244170.0000 - precision0.1: 0.4428 - rec
all0.1: 0.9177 - tp0.3: 2397859.0000 - fp0.3: 1591770.0000 - tn0.3: 15051718.0000 - fn0.3: 570301.0000 - precision0.3: 0.6010 - recall0.3: 0.8079 - tp0.5: 2062821.0000 - fp0.5: 894260.0000
 - tn0.5: 15749228.0000 - fn0.5: 905339.0000 - precision0.5: 0.6976 - recall0.5: 0.6950 - tp0.7: 1513878.0000 - fp0.7: 385604.0000 - tn0.7: 16257884.0000 - fn0.7: 1454282.0000 - precision0
.7: 0.7970 - recall0.7: 0.5100 - tp0.9: 521178.0000 - fp0.9: 63769.0000 - tn0.9: 16579719.0000 - fn0.9: 2446982.0000 - precision0.9: 0.8910 - recall0.9: 0.1756 - accuracy: 0.9082 - auc: 0.
9187 - f1: 0.2629 - val_loss: 0.5978 - val_tp0.1: 736458.0000 - val_fp0.1: 812942.0000 - val_tn0.1: 3258937.0000 - val_fn0.1: 65903.0000 - val_precision0.1: 0.4753 - val_recall0.1: 0.9179
- val_tp0.3: 673047.0000 - val_fp0.3: 451203.0000 - val_tn0.3: 3620676.0000 - val_fn0.3: 129314.0000 - val_precision0.3: 0.5987 - val_recall0.3: 0.8388 - val_tp0.5: 622273.0000 - val_fp0.5
: 311162.0000 - val_tn0.5: 3760717.0000 - val_fn0.5: 180088.0000 - val_precision0.5: 0.6666 - val_recall0.5: 0.7756 - val_tp0.7: 543103.0000 - val_fp0.7: 183256.0000 - val_tn0.7: 3888623.0
000 - val_fn0.7: 259258.0000 - val_precision0.7: 0.7477 - val_recall0.7: 0.6769 - val_tp0.9: 367886.0000 - val_fp0.9: 60748.0000 - val_tn0.9: 4011131.0000 - val_fn0.9: 434475.0000 - val_pr
ecision0.9: 0.8583 - val_recall0.9: 0.4585 - val_accuracy: 0.8992 - val_auc: 0.9218 - val_f1: 0.2827
Epoch 7/20
342/342 [==============================] - 30s 87ms/step - loss: 0.5618 - tp0.1: 2753144.0000 - fp0.1: 3391488.0000 - tn0.1: 13244099.0000 - fn0.1: 222917.0000 - precision0.1: 0.4481 - rec
all0.1: 0.9251 - tp0.3: 2430470.0000 - fp0.3: 1601144.0000 - tn0.3: 15034443.0000 - fn0.3: 545591.0000 - precision0.3: 0.6029 - recall0.3: 0.8167 - tp0.5: 2105059.0000 - fp0.5: 906072.0000
 - tn0.5: 15729515.0000 - fn0.5: 871002.0000 - precision0.5: 0.6991 - recall0.5: 0.7073 - tp0.7: 1566971.0000 - fp0.7: 391164.0000 - tn0.7: 16244423.0000 - fn0.7: 1409090.0000 - precision0
.7: 0.8002 - recall0.7: 0.5265 - tp0.9: 568695.0000 - fp0.9: 65221.0000 - tn0.9: 16570366.0000 - fn0.9: 2407366.0000 - precision0.9: 0.8971 - recall0.9: 0.1911 - accuracy: 0.9094 - auc: 0.
9229 - f1: 0.2635 - val_loss: 0.5711 - val_tp0.1: 744203.0000 - val_fp0.1: 929725.0000 - val_tn0.1: 3153721.0000 - val_fn0.1: 46591.0000 - val_precision0.1: 0.4446 - val_recall0.1: 0.9411
- val_tp0.3: 640265.0000 - val_fp0.3: 353301.0000 - val_tn0.3: 3730145.0000 - val_fn0.3: 150529.0000 - val_precision0.3: 0.6444 - val_recall0.3: 0.8096 - val_tp0.5: 490709.0000 - val_fp0.5
: 135186.0000 - val_tn0.5: 3948260.0000 - val_fn0.5: 300085.0000 - val_precision0.5: 0.7840 - val_recall0.5: 0.6205 - val_tp0.7: 277364.0000 - val_fp0.7: 34487.0000 - val_tn0.7: 4048959.00
00 - val_fn0.7: 513430.0000 - val_precision0.7: 0.8894 - val_recall0.7: 0.3507 - val_tp0.9: 24614.0000 - val_fp0.9: 1054.0000 - val_tn0.9: 4082392.0000 - val_fn0.9: 766180.0000 - val_preci
sion0.9: 0.9589 - val_recall0.9: 0.0311 - val_accuracy: 0.9107 - val_auc: 0.9298 - val_f1: 0.2792
Epoch 8/20
342/342 [==============================] - 28s 81ms/step - loss: 0.5408 - tp0.1: 2752489.0000 - fp0.1: 3321723.0000 - tn0.1: 13326977.0000 - fn0.1: 210459.0000 - precision0.1: 0.4531 - rec
all0.1: 0.9290 - tp0.3: 2438347.0000 - fp0.3: 1518540.0000 - tn0.3: 15130160.0000 - fn0.3: 524601.0000 - precision0.3: 0.6162 - recall0.3: 0.8229 - tp0.5: 2124376.0000 - fp0.5: 851682.0000
 - tn0.5: 15797018.0000 - fn0.5: 838572.0000 - precision0.5: 0.7138 - recall0.5: 0.7170 - tp0.7: 1623433.0000 - fp0.7: 378788.0000 - tn0.7: 16269912.0000 - fn0.7: 1339515.0000 - precision0
.7: 0.8108 - recall0.7: 0.5479 - tp0.9: 664330.0000 - fp0.9: 71506.0000 - tn0.9: 16577194.0000 - fn0.9: 2298618.0000 - precision0.9: 0.9028 - recall0.9: 0.2242 - accuracy: 0.9138 - auc: 0.
9275 - f1: 0.2625 - val_loss: 0.5804 - val_tp0.1: 719809.0000 - val_fp0.1: 677691.0000 - val_tn0.1: 3399808.0000 - val_fn0.1: 76932.0000 - val_precision0.1: 0.5151 - val_recall0.1: 0.9034
- val_tp0.3: 629328.0000 - val_fp0.3: 315236.0000 - val_tn0.3: 3762263.0000 - val_fn0.3: 167413.0000 - val_precision0.3: 0.6663 - val_recall0.3: 0.7899 - val_tp0.5: 549596.0000 - val_fp0.5
: 183412.0000 - val_tn0.5: 3894087.0000 - val_fn0.5: 247145.0000 - val_precision0.5: 0.7498 - val_recall0.5: 0.6898 - val_tp0.7: 420608.0000 - val_fp0.7: 82009.0000 - val_tn0.7: 3995490.00
00 - val_fn0.7: 376133.0000 - val_precision0.7: 0.8368 - val_recall0.7: 0.5279 - val_tp0.9: 188872.0000 - val_fp0.9: 12738.0000 - val_tn0.9: 4064761.0000 - val_fn0.9: 607869.0000 - val_pre
cision0.9: 0.9368 - val_recall0.9: 0.2371 - val_accuracy: 0.9117 - val_auc: 0.9182 - val_f1: 0.2810
Epoch 9/20
342/342 [==============================] - 31s 90ms/step - loss: 0.5325 - tp0.1: 2770579.0000 - fp0.1: 3319105.0000 - tn0.1: 13319456.0000 - fn0.1: 202508.0000 - precision0.1: 0.4550 - rec
all0.1: 0.9319 - tp0.3: 2466041.0000 - fp0.3: 1533716.0000 - tn0.3: 15104845.0000 - fn0.3: 507046.0000 - precision0.3: 0.6165 - recall0.3: 0.8295 - tp0.5: 2177107.0000 - fp0.5: 886546.0000
 - tn0.5: 15752015.0000 - fn0.5: 795980.0000 - precision0.5: 0.7106 - recall0.5: 0.7323 - tp0.7: 1685024.0000 - fp0.7: 402104.0000 - tn0.7: 16236457.0000 - fn0.7: 1288063.0000 - precision0
.7: 0.8073 - recall0.7: 0.5668 - tp0.9: 684970.0000 - fp0.9: 71002.0000 - tn0.9: 16567559.0000 - fn0.9: 2288117.0000 - precision0.9: 0.9061 - recall0.9: 0.2304 - accuracy: 0.9142 - auc: 0.
9299 - f1: 0.2633 - val_loss: 0.6069 - val_tp0.1: 776068.0000 - val_fp0.1: 1351519.0000 - val_tn0.1: 2726452.0000 - val_fn0.1: 20201.0000 - val_precision0.1: 0.3648 - val_recall0.1: 0.9746
 - val_tp0.3: 720731.0000 - val_fp0.3: 628859.0000 - val_tn0.3: 3449112.0000 - val_fn0.3: 75538.0000 - val_precision0.3: 0.5340 - val_recall0.3: 0.9051 - val_tp0.5: 648641.0000 - val_fp0.5
: 344147.0000 - val_tn0.5: 3733824.0000 - val_fn0.5: 147628.0000 - val_precision0.5: 0.6534 - val_recall0.5: 0.8146 - val_tp0.7: 538210.0000 - val_fp0.7: 168258.0000 - val_tn0.7: 3909713.0
000 - val_fn0.7: 258059.0000 - val_precision0.7: 0.7618 - val_recall0.7: 0.6759 - val_tp0.9: 289556.0000 - val_fp0.9: 34074.0000 - val_tn0.9: 4043897.0000 - val_fn0.9: 506713.0000 - val_pr
ecision0.9: 0.8947 - val_recall0.9: 0.3636 - val_accuracy: 0.8991 - val_auc: 0.9424 - val_f1: 0.2808
Epoch 10/20
342/342 [==============================] - 29s 86ms/step - loss: 0.5031 - tp0.1: 2780826.0000 - fp0.1: 3016769.0000 - tn0.1: 13618075.0000 - fn0.1: 195978.0000 - precision0.1: 0.4797 - rec
all0.1: 0.9342 - tp0.3: 2488133.0000 - fp0.3: 1399371.0000 - tn0.3: 15235473.0000 - fn0.3: 488671.0000 - precision0.3: 0.6400 - recall0.3: 0.8358 - tp0.5: 2202797.0000 - fp0.5: 811436.0000
 - tn0.5: 15823408.0000 - fn0.5: 774007.0000 - precision0.5: 0.7308 - recall0.5: 0.7400 - tp0.7: 1728965.0000 - fp0.7: 377368.0000 - tn0.7: 16257476.0000 - fn0.7: 1247839.0000 - precision0
.7: 0.8208 - recall0.7: 0.5808 - tp0.9: 720528.0000 - fp0.9: 68075.0000 - tn0.9: 16566769.0000 - fn0.9: 2256276.0000 - precision0.9: 0.9137 - recall0.9: 0.2420 - accuracy: 0.9192 - auc: 0.
9351 - f1: 0.2636 - val_loss: 0.5387 - val_tp0.1: 760105.0000 - val_fp0.1: 938438.0000 - val_tn0.1: 3135229.0000 - val_fn0.1: 40468.0000 - val_precision0.1: 0.4475 - val_recall0.1: 0.9495
- val_tp0.3: 691484.0000 - val_fp0.3: 448766.0000 - val_tn0.3: 3624901.0000 - val_fn0.3: 109089.0000 - val_precision0.3: 0.6064 - val_recall0.3: 0.8637 - val_tp0.5: 628663.0000 - val_fp0.5
: 268056.0000 - val_tn0.5: 3805611.0000 - val_fn0.5: 171910.0000 - val_precision0.5: 0.7011 - val_recall0.5: 0.7853 - val_tp0.7: 517637.0000 - val_fp0.7: 128039.0000 - val_tn0.7: 3945628.0
000 - val_fn0.7: 282936.0000 - val_precision0.7: 0.8017 - val_recall0.7: 0.6466 - val_tp0.9: 267423.0000 - val_fp0.9: 24096.0000 - val_tn0.9: 4049571.0000 - val_fn0.9: 533150.0000 - val_pr
ecision0.9: 0.9173 - val_recall0.9: 0.3340 - val_accuracy: 0.9097 - val_auc: 0.9378 - val_f1: 0.2821
Epoch 11/20
342/342 [==============================] - 30s 88ms/step - loss: 0.5001 - tp0.1: 2760834.0000 - fp0.1: 2989115.0000 - tn0.1: 13662862.0000 - fn0.1: 198837.0000 - precision0.1: 0.4801 - rec
all0.1: 0.9328 - tp0.3: 2478896.0000 - fp0.3: 1396965.0000 - tn0.3: 15255012.0000 - fn0.3: 480775.0000 - precision0.3: 0.6396 - recall0.3: 0.8376 - tp0.5: 2214110.0000 - fp0.5: 817049.0000
 - tn0.5: 15834928.0000 - fn0.5: 745561.0000 - precision0.5: 0.7304 - recall0.5: 0.7481 - tp0.7: 1770056.0000 - fp0.7: 384025.0000 - tn0.7: 16267952.0000 - fn0.7: 1189615.0000 - precision0
.7: 0.8217 - recall0.7: 0.5981 - tp0.9: 798931.0000 - fp0.9: 75031.0000 - tn0.9: 16576946.0000 - fn0.9: 2160740.0000 - precision0.9: 0.9141 - recall0.9: 0.2699 - accuracy: 0.9203 - auc: 0.
9351 - f1: 0.2623 - val_loss: 0.5461 - val_tp0.1: 730695.0000 - val_fp0.1: 658715.0000 - val_tn0.1: 3417454.0000 - val_fn0.1: 67376.0000 - val_precision0.1: 0.5259 - val_recall0.1: 0.9156
- val_tp0.3: 621793.0000 - val_fp0.3: 261105.0000 - val_tn0.3: 3815064.0000 - val_fn0.3: 176278.0000 - val_precision0.3: 0.7043 - val_recall0.3: 0.7791 - val_tp0.5: 521488.0000 - val_fp0.5
: 133143.0000 - val_tn0.5: 3943026.0000 - val_fn0.5: 276583.0000 - val_precision0.5: 0.7966 - val_recall0.5: 0.6534 - val_tp0.7: 362988.0000 - val_fp0.7: 49452.0000 - val_tn0.7: 4026717.00
00 - val_fn0.7: 435083.0000 - val_precision0.7: 0.8801 - val_recall0.7: 0.4548 - val_tp0.9: 99821.0000 - val_fp0.9: 3904.0000 - val_tn0.9: 4072265.0000 - val_fn0.9: 698250.0000 - val_preci
sion0.9: 0.9624 - val_recall0.9: 0.1251 - val_accuracy: 0.9159 - val_auc: 0.9274 - val_f1: 0.2814
Epoch 12/20
342/342 [==============================] - 30s 87ms/step - loss: 0.4965 - tp0.1: 2780843.0000 - fp0.1: 3042674.0000 - tn0.1: 13597784.0000 - fn0.1: 190347.0000 - precision0.1: 0.4775 - rec
all0.1: 0.9359 - tp0.3: 2483369.0000 - fp0.3: 1382732.0000 - tn0.3: 15257726.0000 - fn0.3: 487821.0000 - precision0.3: 0.6423 - recall0.3: 0.8358 - tp0.5: 2206966.0000 - fp0.5: 794121.0000
 - tn0.5: 15846337.0000 - fn0.5: 764224.0000 - precision0.5: 0.7354 - recall0.5: 0.7428 - tp0.7: 1750172.0000 - fp0.7: 362712.0000 - tn0.7: 16277746.0000 - fn0.7: 1221018.0000 - precision0
.7: 0.8283 - recall0.7: 0.5890 - tp0.9: 762690.0000 - fp0.9: 68788.0000 - tn0.9: 16571670.0000 - fn0.9: 2208500.0000 - precision0.9: 0.9173 - recall0.9: 0.2567 - accuracy: 0.9205 - auc: 0.
9366 - f1: 0.2631 - val_loss: 0.5341 - val_tp0.1: 758000.0000 - val_fp0.1: 932695.0000 - val_tn0.1: 3144776.0000 - val_fn0.1: 38769.0000 - val_precision0.1: 0.4483 - val_recall0.1: 0.9513
- val_tp0.3: 691823.0000 - val_fp0.3: 454271.0000 - val_tn0.3: 3623200.0000 - val_fn0.3: 104946.0000 - val_precision0.3: 0.6036 - val_recall0.3: 0.8683 - val_tp0.5: 633340.0000 - val_fp0.5
: 278624.0000 - val_tn0.5: 3798847.0000 - val_fn0.5: 163429.0000 - val_precision0.5: 0.6945 - val_recall0.5: 0.7949 - val_tp0.7: 534109.0000 - val_fp0.7: 146703.0000 - val_tn0.7: 3930768.0
000 - val_fn0.7: 262660.0000 - val_precision0.7: 0.7845 - val_recall0.7: 0.6703 - val_tp0.9: 318438.0000 - val_fp0.9: 38771.0000 - val_tn0.9: 4038700.0000 - val_fn0.9: 478331.0000 - val_pr
ecision0.9: 0.8915 - val_recall0.9: 0.3997 - val_accuracy: 0.9093 - val_auc: 0.9388 - val_f1: 0.2810
Epoch 13/20
342/342 [==============================] - 29s 84ms/step - loss: 0.4892 - tp0.1: 2779063.0000 - fp0.1: 2979477.0000 - tn0.1: 13665550.0000 - fn0.1: 187558.0000 - precision0.1: 0.4826 - rec
all0.1: 0.9368 - tp0.3: 2496191.0000 - fp0.3: 1375175.0000 - tn0.3: 15269852.0000 - fn0.3: 470430.0000 - precision0.3: 0.6448 - recall0.3: 0.8414 - tp0.5: 2236501.0000 - fp0.5: 810489.0000
 - tn0.5: 15834538.0000 - fn0.5: 730120.0000 - precision0.5: 0.7340 - recall0.5: 0.7539 - tp0.7: 1787849.0000 - fp0.7: 380585.0000 - tn0.7: 16264442.0000 - fn0.7: 1178772.0000 - precision0
.7: 0.8245 - recall0.7: 0.6027 - tp0.9: 798725.0000 - fp0.9: 77694.0000 - tn0.9: 16567333.0000 - fn0.9: 2167896.0000 - precision0.9: 0.9114 - recall0.9: 0.2692 - accuracy: 0.9214 - auc: 0.
9378 - f1: 0.2628 - val_loss: 0.5294 - val_tp0.1: 755023.0000 - val_fp0.1: 867873.0000 - val_tn0.1: 3206834.0000 - val_fn0.1: 44510.0000 - val_precision0.1: 0.4652 - val_recall0.1: 0.9443
- val_tp0.3: 673836.0000 - val_fp0.3: 383349.0000 - val_tn0.3: 3691358.0000 - val_fn0.3: 125697.0000 - val_precision0.3: 0.6374 - val_recall0.3: 0.8428 - val_tp0.5: 597569.0000 - val_fp0.5
: 210770.0000 - val_tn0.5: 3863937.0000 - val_fn0.5: 201964.0000 - val_precision0.5: 0.7393 - val_recall0.5: 0.7474 - val_tp0.7: 457652.0000 - val_fp0.7: 91160.0000 - val_tn0.7: 3983547.00
00 - val_fn0.7: 341881.0000 - val_precision0.7: 0.8339 - val_recall0.7: 0.5724 - val_tp0.9: 170366.0000 - val_fp0.9: 11587.0000 - val_tn0.9: 4063120.0000 - val_fn0.9: 629167.0000 - val_pre
cision0.9: 0.9363 - val_recall0.9: 0.2131 - val_accuracy: 0.9153 - val_auc: 0.9368 - val_f1: 0.2818
Epoch 14/20
342/342 [==============================] - 30s 87ms/step - loss: 0.4909 - tp0.1: 2796281.0000 - fp0.1: 2993242.0000 - tn0.1: 13633299.0000 - fn0.1: 188826.0000 - precision0.1: 0.4830 - rec
all0.1: 0.9367 - tp0.3: 2509770.0000 - fp0.3: 1365834.0000 - tn0.3: 15260707.0000 - fn0.3: 475337.0000 - precision0.3: 0.6476 - recall0.3: 0.8408 - tp0.5: 2246662.0000 - fp0.5: 798278.0000
 - tn0.5: 15828263.0000 - fn0.5: 738445.0000 - precision0.5: 0.7378 - recall0.5: 0.7526 - tp0.7: 1791881.0000 - fp0.7: 369480.0000 - tn0.7: 16257061.0000 - fn0.7: 1193226.0000 - precision0
.7: 0.8291 - recall0.7: 0.6003 - tp0.9: 775898.0000 - fp0.9: 73458.0000 - tn0.9: 16553083.0000 - fn0.9: 2209209.0000 - precision0.9: 0.9135 - recall0.9: 0.2599 - accuracy: 0.9216 - auc: 0.
9379 - f1: 0.2642 - val_loss: 0.5282 - val_tp0.1: 755947.0000 - val_fp0.1: 899285.0000 - val_tn0.1: 3177862.0000 - val_fn0.1: 41146.0000 - val_precision0.1: 0.4567 - val_recall0.1: 0.9484
- val_tp0.3: 680024.0000 - val_fp0.3: 410533.0000 - val_tn0.3: 3666614.0000 - val_fn0.3: 117069.0000 - val_precision0.3: 0.6236 - val_recall0.3: 0.8531 - val_tp0.5: 611482.0000 - val_fp0.5
: 236410.0000 - val_tn0.5: 3840737.0000 - val_fn0.5: 185611.0000 - val_precision0.5: 0.7212 - val_recall0.5: 0.7671 - val_tp0.7: 485673.0000 - val_fp0.7: 108280.0000 - val_tn0.7: 3968867.0
000 - val_fn0.7: 311420.0000 - val_precision0.7: 0.8177 - val_recall0.7: 0.6093 - val_tp0.9: 216751.0000 - val_fp0.9: 17398.0000 - val_tn0.9: 4059749.0000 - val_fn0.9: 580342.0000 - val_pr
ecision0.9: 0.9257 - val_recall0.9: 0.2719 - val_accuracy: 0.9134 - val_auc: 0.9380 - val_f1: 0.2811
Epoch 15/20
342/342 [==============================] - 27s 80ms/step - loss: 0.4895 - tp0.1: 2778517.0000 - fp0.1: 2946222.0000 - tn0.1: 13695622.0000 - fn0.1: 191287.0000 - precision0.1: 0.4854 - rec
all0.1: 0.9356 - tp0.3: 2488194.0000 - fp0.3: 1341585.0000 - tn0.3: 15300259.0000 - fn0.3: 481610.0000 - precision0.3: 0.6497 - recall0.3: 0.8378 - tp0.5: 2226779.0000 - fp0.5: 783374.0000
 - tn0.5: 15858470.0000 - fn0.5: 743025.0000 - precision0.5: 0.7398 - recall0.5: 0.7498 - tp0.7: 1768572.0000 - fp0.7: 363199.0000 - tn0.7: 16278645.0000 - fn0.7: 1201232.0000 - precision0
.7: 0.8296 - recall0.7: 0.5955 - tp0.9: 771789.0000 - fp0.9: 70077.0000 - tn0.9: 16571767.0000 - fn0.9: 2198015.0000 - precision0.9: 0.9168 - recall0.9: 0.2599 - accuracy: 0.9222 - auc: 0.
9375 - f1: 0.2630 - val_loss: 0.5272 - val_tp0.1: 756550.0000 - val_fp0.1: 886258.0000 - val_tn0.1: 3189329.0000 - val_fn0.1: 42103.0000 - val_precision0.1: 0.4605 - val_recall0.1: 0.9473
- val_tp0.3: 683180.0000 - val_fp0.3: 415168.0000 - val_tn0.3: 3660419.0000 - val_fn0.3: 115473.0000 - val_precision0.3: 0.6220 - val_recall0.3: 0.8554 - val_tp0.5: 618115.0000 - val_fp0.5
: 244352.0000 - val_tn0.5: 3831235.0000 - val_fn0.5: 180538.0000 - val_precision0.5: 0.7167 - val_recall0.5: 0.7739 - val_tp0.7: 498018.0000 - val_fp0.7: 115297.0000 - val_tn0.7: 3960290.0
000 - val_fn0.7: 300635.0000 - val_precision0.7: 0.8120 - val_recall0.7: 0.6236 - val_tp0.9: 233878.0000 - val_fp0.9: 19740.0000 - val_tn0.9: 4055847.0000 - val_fn0.9: 564775.0000 - val_pr
ecision0.9: 0.9222 - val_recall0.9: 0.2928 - val_accuracy: 0.9128 - val_auc: 0.9378 - val_f1: 0.2816
Epoch 16/20
342/342 [==============================] - 29s 86ms/step - loss: 0.4848 - tp0.1: 2781482.0000 - fp0.1: 2957469.0000 - tn0.1: 13686899.0000 - fn0.1: 185798.0000 - precision0.1: 0.4847 - rec
all0.1: 0.9374 - tp0.3: 2494189.0000 - fp0.3: 1353502.0000 - tn0.3: 15290866.0000 - fn0.3: 473091.0000 - precision0.3: 0.6482 - recall0.3: 0.8406 - tp0.5: 2230152.0000 - fp0.5: 792220.0000
 - tn0.5: 15852148.0000 - fn0.5: 737128.0000 - precision0.5: 0.7379 - recall0.5: 0.7516 - tp0.7: 1783255.0000 - fp0.7: 363318.0000 - tn0.7: 16281050.0000 - fn0.7: 1184025.0000 - precision0
.7: 0.8307 - recall0.7: 0.6010 - tp0.9: 789040.0000 - fp0.9: 65516.0000 - tn0.9: 16578852.0000 - fn0.9: 2178240.0000 - precision0.9: 0.9233 - recall0.9: 0.2659 - accuracy: 0.9220 - auc: 0.
9387 - f1: 0.2628 - val_loss: 0.5289 - val_tp0.1: 757278.0000 - val_fp0.1: 905312.0000 - val_tn0.1: 3170851.0000 - val_fn0.1: 40799.0000 - val_precision0.1: 0.4555 - val_recall0.1: 0.9489
- val_tp0.3: 686534.0000 - val_fp0.3: 428363.0000 - val_tn0.3: 3647800.0000 - val_fn0.3: 111543.0000 - val_precision0.3: 0.6158 - val_recall0.3: 0.8602 - val_tp0.5: 623130.0000 - val_fp0.5
: 254259.0000 - val_tn0.5: 3821904.0000 - val_fn0.5: 174947.0000 - val_precision0.5: 0.7102 - val_recall0.5: 0.7808 - val_tp0.7: 505552.0000 - val_fp0.7: 121120.0000 - val_tn0.7: 3955043.0
000 - val_fn0.7: 292525.0000 - val_precision0.7: 0.8067 - val_recall0.7: 0.6335 - val_tp0.9: 244234.0000 - val_fp0.9: 21695.0000 - val_tn0.9: 4054468.0000 - val_fn0.9: 553843.0000 - val_pr
ecision0.9: 0.9184 - val_recall0.9: 0.3060 - val_accuracy: 0.9119 - val_auc: 0.9382 - val_f1: 0.2814
Epoch 17/20
342/342 [==============================] - 30s 86ms/step - loss: 0.4849 - tp0.1: 2790598.0000 - fp0.1: 2957998.0000 - tn0.1: 13679248.0000 - fn0.1: 183804.0000 - precision0.1: 0.4854 - rec
all0.1: 0.9382 - tp0.3: 2505785.0000 - fp0.3: 1360823.0000 - tn0.3: 15276423.0000 - fn0.3: 468617.0000 - precision0.3: 0.6481 - recall0.3: 0.8425 - tp0.5: 2243181.0000 - fp0.5: 797599.0000
 - tn0.5: 15839647.0000 - fn0.5: 731221.0000 - precision0.5: 0.7377 - recall0.5: 0.7542 - tp0.7: 1792202.0000 - fp0.7: 377983.0000 - tn0.7: 16259263.0000 - fn0.7: 1182200.0000 - precision0
.7: 0.8258 - recall0.7: 0.6025 - tp0.9: 784914.0000 - fp0.9: 75124.0000 - tn0.9: 16562122.0000 - fn0.9: 2189488.0000 - precision0.9: 0.9127 - recall0.9: 0.2639 - accuracy: 0.9220 - auc: 0.
9387 - f1: 0.2634 - val_loss: 0.5301 - val_tp0.1: 751131.0000 - val_fp0.1: 924609.0000 - val_tn0.1: 3159383.0000 - val_fn0.1: 39117.0000 - val_precision0.1: 0.4482 - val_recall0.1: 0.9505
- val_tp0.3: 682847.0000 - val_fp0.3: 443618.0000 - val_tn0.3: 3640374.0000 - val_fn0.3: 107401.0000 - val_precision0.3: 0.6062 - val_recall0.3: 0.8641 - val_tp0.5: 620126.0000 - val_fp0.5
: 265041.0000 - val_tn0.5: 3818951.0000 - val_fn0.5: 170122.0000 - val_precision0.5: 0.7006 - val_recall0.5: 0.7847 - val_tp0.7: 505243.0000 - val_fp0.7: 127349.0000 - val_tn0.7: 3956643.0
000 - val_fn0.7: 285005.0000 - val_precision0.7: 0.7987 - val_recall0.7: 0.6393 - val_tp0.9: 248302.0000 - val_fp0.9: 23740.0000 - val_tn0.9: 4060252.0000 - val_fn0.9: 541946.0000 - val_pr
ecision0.9: 0.9127 - val_recall0.9: 0.3142 - val_accuracy: 0.9107 - val_auc: 0.9382 - val_f1: 0.2790
Epoch 18/20
342/342 [==============================] - 30s 88ms/step - loss: 0.4853 - tp0.1: 2805900.0000 - fp0.1: 2941444.0000 - tn0.1: 13677897.0000 - fn0.1: 186407.0000 - precision0.1: 0.4882 - rec
all0.1: 0.9377 - tp0.3: 2517150.0000 - fp0.3: 1335602.0000 - tn0.3: 15283739.0000 - fn0.3: 475157.0000 - precision0.3: 0.6533 - recall0.3: 0.8412 - tp0.5: 2248387.0000 - fp0.5: 781307.0000
 - tn0.5: 15838034.0000 - fn0.5: 743920.0000 - precision0.5: 0.7421 - recall0.5: 0.7514 - tp0.7: 1790819.0000 - fp0.7: 356494.0000 - tn0.7: 16262847.0000 - fn0.7: 1201488.0000 - precision0
.7: 0.8340 - recall0.7: 0.5985 - tp0.9: 783564.0000 - fp0.9: 70623.0000 - tn0.9: 16548718.0000 - fn0.9: 2208743.0000 - precision0.9: 0.9173 - recall0.9: 0.2619 - accuracy: 0.9222 - auc: 0.
9390 - f1: 0.2648 - val_loss: 0.5309 - val_tp0.1: 759007.0000 - val_fp0.1: 906199.0000 - val_tn0.1: 3168039.0000 - val_fn0.1: 40995.0000 - val_precision0.1: 0.4558 - val_recall0.1: 0.9488
- val_tp0.3: 688612.0000 - val_fp0.3: 432765.0000 - val_tn0.3: 3641473.0000 - val_fn0.3: 111390.0000 - val_precision0.3: 0.6141 - val_recall0.3: 0.8608 - val_tp0.5: 625381.0000 - val_fp0.5
: 258408.0000 - val_tn0.5: 3815830.0000 - val_fn0.5: 174621.0000 - val_precision0.5: 0.7076 - val_recall0.5: 0.7817 - val_tp0.7: 508544.0000 - val_fp0.7: 124098.0000 - val_tn0.7: 3950140.0
000 - val_fn0.7: 291458.0000 - val_precision0.7: 0.8038 - val_recall0.7: 0.6357 - val_tp0.9: 247680.0000 - val_fp0.9: 22653.0000 - val_tn0.9: 4051585.0000 - val_fn0.9: 552322.0000 - val_pr
ecision0.9: 0.9162 - val_recall0.9: 0.3096 - val_accuracy: 0.9112 - val_auc: 0.9378 - val_f1: 0.2820
Epoch 19/20
342/342 [==============================] - 30s 87ms/step - loss: 0.4853 - tp0.1: 2765154.0000 - fp0.1: 2965014.0000 - tn0.1: 13695648.0000 - fn0.1: 185832.0000 - precision0.1: 0.4826 - rec
all0.1: 0.9370 - tp0.3: 2489223.0000 - fp0.3: 1363429.0000 - tn0.3: 15297233.0000 - fn0.3: 461763.0000 - precision0.3: 0.6461 - recall0.3: 0.8435 - tp0.5: 2236269.0000 - fp0.5: 805108.0000
 - tn0.5: 15855554.0000 - fn0.5: 714717.0000 - precision0.5: 0.7353 - recall0.5: 0.7578 - tp0.7: 1791573.0000 - fp0.7: 372427.0000 - tn0.7: 16288235.0000 - fn0.7: 1159413.0000 - precision0
.7: 0.8279 - recall0.7: 0.6071 - tp0.9: 777165.0000 - fp0.9: 72719.0000 - tn0.9: 16587943.0000 - fn0.9: 2173821.0000 - precision0.9: 0.9144 - recall0.9: 0.2634 - accuracy: 0.9225 - auc: 0.
9385 - f1: 0.2616 - val_loss: 0.5302 - val_tp0.1: 757018.0000 - val_fp0.1: 914987.0000 - val_tn0.1: 3162308.0000 - val_fn0.1: 39927.0000 - val_precision0.1: 0.4528 - val_recall0.1: 0.9499
- val_tp0.3: 687887.0000 - val_fp0.3: 439024.0000 - val_tn0.3: 3638271.0000 - val_fn0.3: 109058.0000 - val_precision0.3: 0.6104 - val_recall0.3: 0.8632 - val_tp0.5: 624960.0000 - val_fp0.5
: 262546.0000 - val_tn0.5: 3814749.0000 - val_fn0.5: 171985.0000 - val_precision0.5: 0.7042 - val_recall0.5: 0.7842 - val_tp0.7: 509687.0000 - val_fp0.7: 126771.0000 - val_tn0.7: 3950524.0
000 - val_fn0.7: 287258.0000 - val_precision0.7: 0.8008 - val_recall0.7: 0.6396 - val_tp0.9: 251547.0000 - val_fp0.9: 23931.0000 - val_tn0.9: 4053364.0000 - val_fn0.9: 545398.0000 - val_pr
ecision0.9: 0.9131 - val_recall0.9: 0.3156 - val_accuracy: 0.9109 - val_auc: 0.9382 - val_f1: 0.2811
Epoch 20/20
342/342 [==============================] - 29s 84ms/step - loss: 0.4987 - tp0.1: 2765147.0000 - fp0.1: 2972400.0000 - tn0.1: 13672870.0000 - fn0.1: 201231.0000 - precision0.1: 0.4819 - rec
all0.1: 0.9322 - tp0.3: 2475699.0000 - fp0.3: 1368612.0000 - tn0.3: 15276658.0000 - fn0.3: 490679.0000 - precision0.3: 0.6440 - recall0.3: 0.8346 - tp0.5: 2217066.0000 - fp0.5: 801284.0000
 - tn0.5: 15843986.0000 - fn0.5: 749312.0000 - precision0.5: 0.7345 - recall0.5: 0.7474 - tp0.7: 1769267.0000 - fp0.7: 376020.0000 - tn0.7: 16269250.0000 - fn0.7: 1197111.0000 - precision0
.7: 0.8247 - recall0.7: 0.5964 - tp0.9: 778944.0000 - fp0.9: 72985.0000 - tn0.9: 16572285.0000 - fn0.9: 2187434.0000 - precision0.9: 0.9143 - recall0.9: 0.2626 - accuracy: 0.9209 - auc: 0.
9352 - f1: 0.2628 - val_loss: 0.5304 - val_tp0.1: 756740.0000 - val_fp0.1: 911039.0000 - val_tn0.1: 3166007.0000 - val_fn0.1: 40454.0000 - val_precision0.1: 0.4537 - val_recall0.1: 0.9493
- val_tp0.3: 686440.0000 - val_fp0.3: 434245.0000 - val_tn0.3: 3642801.0000 - val_fn0.3: 110754.0000 - val_precision0.3: 0.6125 - val_recall0.3: 0.8611 - val_tp0.5: 622523.0000 - val_fp0.5
: 258158.0000 - val_tn0.5: 3818888.0000 - val_fn0.5: 174671.0000 - val_precision0.5: 0.7069 - val_recall0.5: 0.7809 - val_tp0.7: 505054.0000 - val_fp0.7: 122534.0000 - val_tn0.7: 3954512.0
000 - val_fn0.7: 292140.0000 - val_precision0.7: 0.8048 - val_recall0.7: 0.6335 - val_tp0.9: 241926.0000 - val_fp0.9: 21836.0000 - val_tn0.9: 4055210.0000 - val_fn0.9: 555268.0000 - val_pr
ecision0.9: 0.9172 - val_recall0.9: 0.3035 - val_accuracy: 0.9112 - val_auc: 0.9379 - val_f1: 0.2811
--- Running training session 64/140
{'hp_epochs': 20, 'hp_batch_size': 14, 'hp_scaler': 'quant_g', 'hp_n_levels': 7, 'hp_first_filters': 16, 'hp_pool_size': 2, 'hp_input_size': 4096, 'hp_lr_start': 0.027144629354532844, 'hp_
lr_power': 5.0}
--- repeat #: 2
input - shape:   (None, 4096, 1)
output - shape:  (None, 4096, 1)
bash: line 1: 36008 Segmentation fault      python src/fluotracify/training/search_hparams.py --num_session_groups 70 --fluotracify_path /beegfs/ye53nis/drmed-git/src --csv_path_train /bee
gfs/ye53nis/saves/firstartifact_Nov2020_train_max2sets --csv_path_val /beegfs/ye53nis/saves/firstartifact_Nov2020_val_max2sets_SORTEDIN --col_per_example 3
2021/08/08 13:58:32 ERROR mlflow.cli: === Run (ID '0a91e997b0884a00bd6c932104f84722') failed ===
(tf) [ye53nis@node130 drmed-git]$

2.1.3 Run 2 - new hparams

2.1.3.1 Record metadata
  1. Current directory, last 5 git commits 
    pwd
git log -5

    (tf) [ye53nis@node130 drmed-git]$ pwd
/beegfs/ye53nis/drmed-git
(tf) [ye53nis@node130 drmed-git]$ git log -5
commit 150ad647300f3306635fa7f5d75fa0d66165df25
Author: Apoplex <oligolex@vivaldi.net>
Date:   Sun Aug 8 17:28:49 2021 +0200

    Fix missing brackets

commit 6cef8d3f9166fe6cb19b6ec064573907a0ca50a6
Author: Apoplex <oligolex@vivaldi.net>
Date:   Sun Aug 8 17:26:06 2021 +0200

    Adjust batch size hparam to upper limit 30

commit 609a75bccfd08fa36b2d55861ec5d7954e1adbe7
Author: Apoplex <oligolex@vivaldi.net>
Date:   Sun Aug 8 17:05:12 2021 +0200

    Adjust hyperparameters

commit 739b18bf64ef886d7ae0468b4fc3d5b9b4ccf12c
Author: Apoplex <oligolex@vivaldi.net>
Date:   Sun Aug 8 16:42:16 2021 +0200

    Change random seed to enable different 2nd run

commit aa5b9bc35c53c4fd1525c6b812b2a28532ae7afb
Author: Apoplex <oligolex@vivaldi.net>
Date:   Sat Aug 7 22:15:02 2021 +0200

    Add hparams combi restriction; add metadata

    problems arise, if in the random combination of hparams, 2*pool_size**n_levels
    is bigger than the input_size. That's why these cases are skipped now.
(tf) [ye53nis@node130 drmed-git]$
  2. system and conda env didn't change, see above.
2.1.3.2 Mlflow run 2 (failed mid run)
  • The last parent run failed to an unkown error (bash: line 1: 36008 Segmentation fault). That means only 63/140 sessions could be run, while also only 45 sessions were eligible (for the rest, the hparams were not compatible with each other). Still, these first 45 runs delivered a lot of interesting information, so I decided to skim through MLflow to look for improvements in the hparam space and I came up with the following:
    • leave out smallest batch sizes and go from 4 to 30
    • make number of levels an integer interval from 1 to 9
    • make first filters an integer interval from 1 to 128
    • add poolsize (+ stride + kernel size) 8
    • adjust starting learning rate real interval to 1e-6 to 0.06
    • make learning rate power integer interval from 1 to 7
  • now the run: 
    mlflow run . -e search_hparams -P fluotracify_path=/beegfs/ye53nis/drmed-git/src/ -P csv_path_train=/beegfs/ye53nis/saves/firstartifact_Nov2020_train_max2sets -P csv_path_val=/beegfs/ye53nis/saves/firstartifact_Nov2020_val_max2sets_SORTEDIN -P num_session_groups=60

    2021-08-09 03:52:38.082134: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba5fe00 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_f/_843 action_count 94195455584406 step 0 next 151
2021-08-09 03:52:38.082152: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba5ff00 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_t/_844 action_count 94195455584407 step 0 next 2585
2021-08-09 03:52:38.082174: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60000 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_f/_853 action_count 94195455584408 step 0 next 2261
2021-08-09 03:52:38.082198: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60100 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_t/_854 action_count 94195455584409 step 0 next 1893
2021-08-09 03:52:38.082216: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60200 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_f/_871 action_count 94195455584410 step 0 next 2158
2021-08-09 03:52:38.082234: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60300 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_t/_872 action_count 94195455584411 step 0 next 2224
2021-08-09 03:52:38.082258: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60400 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_f/_881 action_count 94195455584412 step 0 next 172
2021-08-09 03:52:38.082277: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60500 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_t/_882 action_count 94195455584413 step 0 next 2654
2021-08-09 03:52:38.082295: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60600 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_f/_899 action_count 94195455584414 step 0 next 1354
2021-08-09 03:52:38.082316: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60700 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_t/_900 action_count 94195455584415 step 0 next 2250
2021-08-09 03:52:38.082338: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60800 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_f/_909 action_count 94195455584416 step 0 next 2458
2021-08-09 03:52:38.082359: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60900 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_t/_910 action_count 94195455584417 step 0 next 1032
2021-08-09 03:52:38.082380: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60a00 of size 256 by op Sub action_count 94195480110816 step 0 next 970
2021-08-09 03:52:38.082401: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60b00 of size 256 by op AssignAddVariableOp_2 action_count 94195467561410 step 17436438550591945738 next 1371
2021-08-09 03:52:38.082422: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60c00 of size 512 by op Fill action_count 94195467562918 step 0 next 2415
2021-08-09 03:52:38.082443: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba60e00 of size 512 by op Fill action_count 94195467562919 step 0 next 2029
2021-08-09 03:52:38.082466: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61000 of size 256 by op AssignAddVariableOp_52 action_count 94195467505890 step 11647544380785529683 next 2491
2021-08-09 03:52:38.082487: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61100 of size 768 by op Fill action_count 94195467562905 step 0 next 1453
2021-08-09 03:52:38.082508: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61400 of size 256 by op AssignAddVariableOp_20 action_count 94195467561512 step 17436438550591945738 next 1396
2021-08-09 03:52:38.082538: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61500 of size 256 by op AssignAddVariableOp_18 action_count 94195467561452 step 17436438550591945738 next 2965
2021-08-09 03:52:38.082558: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61600 of size 256 by op Sub action_count 94195480110817 step 0 next 1920
2021-08-09 03:52:38.082575: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61700 of size 256 by op AssignAddVariableOp_28 action_count 94195467561516 step 17436438550591945738 next 730
2021-08-09 03:52:38.082593: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61800 of size 256 by op AssignAddVariableOp_26 action_count 94195467561464 step 17436438550591945738 next 3037
2021-08-09 03:52:38.082616: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61900 of size 256 by op AssignAddVariableOp_36 action_count 94195467561520 step 17436438550591945738 next 1876
2021-08-09 03:52:38.082636: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61a00 of size 256 by op AssignAddVariableOp_34 action_count 94195467561476 step 17436438550591945738 next 1958
2021-08-09 03:52:38.082657: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba61b00 of size 1536 by op Fill action_count 94195467562929 step 0 next 1889
2021-08-09 03:52:38.082678: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba62100 of size 256 by op Sub action_count 94195299053113 step 0 next 1087
2021-08-09 03:52:38.082698: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba62200 of size 256 by op Sub action_count 94195299053114 step 0 next 680
2021-08-09 03:52:38.082718: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba62300 of size 2048 by op Fill action_count 94195455013702 step 0 next 2691
2021-08-09 03:52:38.082738: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba62b00 of size 2048 by op Fill action_count 94195455013703 step 0 next 3049
2021-08-09 03:52:38.082759: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba63300 of size 2048 by op Fill action_count 94195455013704 step 0 next 2707
2021-08-09 03:52:38.082780: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba63b00 of size 2048 by op Fill action_count 94195455013706 step 0 next 1505
2021-08-09 03:52:38.082801: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64300 of size 256 by op Sub action_count 94195365012693 step 0 next 2311
2021-08-09 03:52:38.082821: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64400 of size 256 by op Sub action_count 94195365012694 step 0 next 638
2021-08-09 03:52:38.082843: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64500 of size 256 by op assert_greater_equal_28/Assert/AssertGuard/pivot_f/_787 action_count 94195455013959 step 0 next 2355
2021-08-09 03:52:38.082866: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64600 of size 256 by op assert_greater_equal_28/Assert/AssertGuard/pivot_t/_788 action_count 94195455013960 step 0 next 2332
2021-08-09 03:52:38.082890: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64700 of size 256 by op assert_less_equal_28/Assert/AssertGuard/pivot_f/_797 action_count 94195455013961 step 0 next 2330
2021-08-09 03:52:38.082910: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64800 of size 256 by op assert_less_equal_28/Assert/AssertGuard/pivot_t/_798 action_count 94195455013962 step 0 next 2402
2021-08-09 03:52:38.082933: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64900 of size 256 by op gradient_tape/unet_depth3/encode0/batch_normalization_546/moments/scalar action_count 94195455013963 step 0 nex
t 114
2021-08-09 03:52:38.082952: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64a00 of size 256 by op assert_greater_equal_29/Assert/AssertGuard/pivot_f/_815 action_count 94195455013964 step 0 next 2297
2021-08-09 03:52:38.082974: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64b00 of size 256 by op assert_greater_equal_29/Assert/AssertGuard/pivot_t/_816 action_count 94195455013965 step 0 next 2106
2021-08-09 03:52:38.082997: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64c00 of size 256 by op Adam/Adam/Const action_count 94195455013966 step 0 next 2308
2021-08-09 03:52:38.083015: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64d00 of size 256 by op assert_less_equal_29/Assert/AssertGuard/pivot_f/_825 action_count 94195455013967 step 0 next 2318
2021-08-09 03:52:38.083036: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64e00 of size 256 by op assert_less_equal_29/Assert/AssertGuard/pivot_t/_826 action_count 94195455013968 step 0 next 2323
2021-08-09 03:52:38.083059: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba64f00 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_f/_843 action_count 94195455013969 step 0 next 2391
2021-08-09 03:52:38.083079: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65000 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_t/_844 action_count 94195455013970 step 0 next 1074
2021-08-09 03:52:38.083097: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65100 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_f/_853 action_count 94195455013971 step 0 next 2382
2021-08-09 03:52:38.083118: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65200 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_t/_854 action_count 94195455013972 step 0 next 2272
2021-08-09 03:52:38.083140: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65300 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_f/_871 action_count 94195455013973 step 0 next 1355
2021-08-09 03:52:38.083158: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65400 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_t/_872 action_count 94195455013974 step 0 next 2378
2021-08-09 03:52:38.083178: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65500 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_f/_881 action_count 94195455013975 step 0 next 2306
2021-08-09 03:52:38.083201: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65600 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_t/_882 action_count 94195455013976 step 0 next 2294
2021-08-09 03:52:38.083220: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65700 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_f/_899 action_count 94195455013977 step 0 next 2331
2021-08-09 03:52:38.083243: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65800 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_t/_900 action_count 94195455013978 step 0 next 2322
2021-08-09 03:52:38.083263: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65900 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_f/_909 action_count 94195455013979 step 0 next 2221
2021-08-09 03:52:38.083283: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65a00 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_t/_910 action_count 94195455013980 step 0 next 2336
2021-08-09 03:52:38.083306: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65b00 of size 256 by op assert_greater_equal_15/Assert/AssertGuard/pivot_f/_423 action_count 94195455584346 step 0 next 2019
2021-08-09 03:52:38.083324: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65c00 of size 256 by op assert_greater_equal_15/Assert/AssertGuard/pivot_t/_424 action_count 94195455584347 step 0 next 2376
2021-08-09 03:52:38.083346: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65d00 of size 256 by op assert_less_equal_15/Assert/AssertGuard/pivot_f/_433 action_count 94195455584348 step 0 next 1688
2021-08-09 03:52:38.083369: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65e00 of size 256 by op assert_less_equal_15/Assert/AssertGuard/pivot_t/_434 action_count 94195455584349 step 0 next 2620
2021-08-09 03:52:38.083391: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba65f00 of size 256 by op assert_greater_equal_16/Assert/AssertGuard/pivot_f/_451 action_count 94195455584350 step 0 next 391
2021-08-09 03:52:38.083414: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66000 of size 256 by op assert_greater_equal_16/Assert/AssertGuard/pivot_t/_452 action_count 94195455584351 step 0 next 1462
2021-08-09 03:52:38.083435: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66100 of size 256 by op assert_less_equal_16/Assert/AssertGuard/pivot_f/_461 action_count 94195455584352 step 0 next 985
2021-08-09 03:52:38.083453: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66200 of size 256 by op assert_less_equal_16/Assert/AssertGuard/pivot_t/_462 action_count 94195455584353 step 0 next 2287
2021-08-09 03:52:38.083475: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66300 of size 256 by op assert_greater_equal_17/Assert/AssertGuard/pivot_f/_479 action_count 94195455584354 step 0 next 2292
2021-08-09 03:52:38.083495: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66400 of size 256 by op assert_greater_equal_17/Assert/AssertGuard/pivot_t/_480 action_count 94195455584355 step 0 next 2350
2021-08-09 03:52:38.083522: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66500 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_f/_489 action_count 94195455584356 step 0 next 2256
2021-08-09 03:52:38.083546: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66600 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_t/_490 action_count 94195455584357 step 0 next 2310
2021-08-09 03:52:38.083566: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66700 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_f/_507 action_count 94195455584358 step 0 next 60
2021-08-09 03:52:38.083587: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66800 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_t/_508 action_count 94195455584359 step 0 next 1380
2021-08-09 03:52:38.083608: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66900 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_f/_517 action_count 94195455584360 step 0 next 2375
2021-08-09 03:52:38.083628: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66a00 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_t/_518 action_count 94195455584361 step 0 next 2497
2021-08-09 03:52:38.083646: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66b00 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_f/_535 action_count 94195455584362 step 0 next 677
2021-08-09 03:52:38.083663: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66c00 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_t/_536 action_count 94195455584363 step 0 next 2326
2021-08-09 03:52:38.083687: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66d00 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_f/_545 action_count 94195455584364 step 0 next 2928
2021-08-09 03:52:38.083707: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66e00 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_t/_546 action_count 94195455584365 step 0 next 2116
2021-08-09 03:52:38.083728: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba66f00 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_f/_563 action_count 94195455584366 step 0 next 411
2021-08-09 03:52:38.083751: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67000 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_t/_564 action_count 94195455584367 step 0 next 1778
2021-08-09 03:52:38.083770: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67100 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_f/_573 action_count 94195455584368 step 0 next 2345
2021-08-09 03:52:38.083791: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67200 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_t/_574 action_count 94195455584369 step 0 next 315
2021-08-09 03:52:38.083810: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67300 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_f/_591 action_count 94195455584370 step 0 next 373
2021-08-09 03:52:38.083831: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67400 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_t/_592 action_count 94195455584371 step 0 next 1535
2021-08-09 03:52:38.083854: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67500 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_f/_601 action_count 94195455584372 step 0 next 1914
2021-08-09 03:52:38.083872: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67600 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_t/_602 action_count 94195455584373 step 0 next 2521
2021-08-09 03:52:38.083894: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67700 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_f/_619 action_count 94195455584374 step 0 next 2110
2021-08-09 03:52:38.083915: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67800 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_t/_620 action_count 94195455584375 step 0 next 968
2021-08-09 03:52:38.083932: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67900 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_f/_629 action_count 94195455584376 step 0 next 2437
2021-08-09 03:52:38.083954: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67a00 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_t/_630 action_count 94195455584377 step 0 next 218
2021-08-09 03:52:38.083975: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67b00 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_f/_647 action_count 94195455584378 step 0 next 1581
2021-08-09 03:52:38.083992: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67c00 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_t/_648 action_count 94195455584379 step 0 next 2337
2021-08-09 03:52:38.084013: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67d00 of size 256 by op Sub action_count 94195299053151 step 0 next 976
2021-08-09 03:52:38.084034: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67e00 of size 256 by op Sub action_count 94195299053152 step 0 next 1112
2021-08-09 03:52:38.084055: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba67f00 of size 2048 by op Fill action_count 94195455013774 step 0 next 1401
2021-08-09 03:52:38.084076: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba68700 of size 2048 by op Fill action_count 94195455013776 step 0 next 1101
2021-08-09 03:52:38.084097: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba68f00 of size 2048 by op Fill action_count 94195455013777 step 0 next 519
2021-08-09 03:52:38.084114: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba69700 of size 2048 by op Fill action_count 94195455013778 step 0 next 799
2021-08-09 03:52:38.084135: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba69f00 of size 3072 by op Fill action_count 94195455013780 step 0 next 607
2021-08-09 03:52:38.084156: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba6ab00 of size 256 by op Sub action_count 94195299053177 step 0 next 2325
2021-08-09 03:52:38.084173: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba6ac00 of size 256 by op Sub action_count 94195299053178 step 0 next 1838
2021-08-09 03:52:38.084194: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abba6ad00 of size 995328 by op Fill action_count 94195455013725 step 0 next 335
2021-08-09 03:52:38.084211: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5dd00 of size 256 by op Sub action_count 94195129464776 step 0 next 1993
2021-08-09 03:52:38.084230: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5de00 of size 4352 by op Fill action_count 94195455013478 step 0 next 1049
2021-08-09 03:52:38.084251: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5ef00 of size 256 by op Sub action_count 94195299052776 step 0 next 2327
2021-08-09 03:52:38.084268: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f000 of size 512 by op Fill action_count 94195455013382 step 0 next 444
2021-08-09 03:52:38.084291: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f200 of size 256 by op AssignAddVariableOp_12 action_count 94195455011283 step 18076379956407770847 next 3070
2021-08-09 03:52:38.084333: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f300 of size 256 by op AssignAddVariableOp_10 action_count 94195455011215 step 18076379956407770847 next 1540
2021-08-09 03:52:38.084353: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f400 of size 256 by op Sub action_count 94195299052789 step 0 next 1949
2021-08-09 03:52:38.084374: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f500 of size 256 by op Sub action_count 94195129464761 step 0 next 988
2021-08-09 03:52:38.084394: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f600 of size 256 by op Sub action_count 94195152301775 step 0 next 1907
2021-08-09 03:52:38.084414: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f700 of size 256 by op Sub action_count 94195152301788 step 0 next 1591
2021-08-09 03:52:38.084430: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f800 of size 256 by op Sub action_count 94195129464762 step 0 next 2090
2021-08-09 03:52:38.084448: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbb5f900 of size 4673536 by op Fill action_count 94195455013689 step 0 next 59
2021-08-09 03:52:38.084468: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd4900 of size 256 by op Sub action_count 94195065726202 step 0 next 659
2021-08-09 03:52:38.084490: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd4a00 of size 256 by op assert_less_equal_16/Assert/AssertGuard/pivot_f/_461 action_count 94195455013907 step 0 next 2217
2021-08-09 03:52:38.084508: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd4b00 of size 256 by op assert_less_equal_16/Assert/AssertGuard/pivot_t/_462 action_count 94195455013908 step 0 next 1616
2021-08-09 03:52:38.084539: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd4c00 of size 256 by op assert_greater_equal_17/Assert/AssertGuard/pivot_f/_479 action_count 94195455013909 step 0 next 2204
2021-08-09 03:52:38.084577: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd4d00 of size 256 by op assert_greater_equal_17/Assert/AssertGuard/pivot_t/_480 action_count 94195455013910 step 0 next 339
2021-08-09 03:52:38.084611: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd4e00 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_f/_489 action_count 94195455013911 step 0 next 701
2021-08-09 03:52:38.084635: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd4f00 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_t/_490 action_count 94195455013912 step 0 next 1708
2021-08-09 03:52:38.084657: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5000 of size 256 by op gradient_tape/unet_depth3/two_conv_center/batch_normalization_552/moments/scalar action_count 94195455013913 st
ep 0 next 903
2021-08-09 03:52:38.084675: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5100 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_f/_507 action_count 94195455013914 step 0 next 2371
2021-08-09 03:52:38.084698: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5200 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_t/_508 action_count 94195455013915 step 0 next 263
2021-08-09 03:52:38.084721: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5300 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_f/_517 action_count 94195455013916 step 0 next 154
2021-08-09 03:52:38.084744: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5400 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_t/_518 action_count 94195455013917 step 0 next 2321
2021-08-09 03:52:38.084765: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5500 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_f/_535 action_count 94195455013918 step 0 next 2253
2021-08-09 03:52:38.084783: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5600 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_t/_536 action_count 94195455013919 step 0 next 2236
2021-08-09 03:52:38.084802: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5700 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_f/_545 action_count 94195455013920 step 0 next 2222
2021-08-09 03:52:38.084823: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5800 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_t/_546 action_count 94195455013921 step 0 next 1262
2021-08-09 03:52:38.084847: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5900 of size 256 by op gradient_tape/unet_depth3/encode2/batch_normalization_551/moments/scalar action_count 94195455013922 step 0 nex
t 2443
2021-08-09 03:52:38.084868: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5a00 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_f/_563 action_count 94195455013923 step 0 next 2313
2021-08-09 03:52:38.084889: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5b00 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_t/_564 action_count 94195455013924 step 0 next 2293
2021-08-09 03:52:38.084912: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5c00 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_f/_573 action_count 94195455013925 step 0 next 2387
2021-08-09 03:52:38.084934: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5d00 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_t/_574 action_count 94195455013926 step 0 next 2328
2021-08-09 03:52:38.084956: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5e00 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_f/_591 action_count 94195455013927 step 0 next 2270
2021-08-09 03:52:38.084977: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd5f00 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_t/_592 action_count 94195455013928 step 0 next 2252
2021-08-09 03:52:38.085000: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6000 of size 256 by op gradient_tape/unet_depth3/encode2/batch_normalization_550/moments/scalar action_count 94195455013929 step 0 nex
t 2392
2021-08-09 03:52:38.085023: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6100 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_f/_601 action_count 94195455013930 step 0 next 2368
2021-08-09 03:52:38.085044: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6200 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_t/_602 action_count 94195455013931 step 0 next 2385
2021-08-09 03:52:38.085067: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6300 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_f/_619 action_count 94195455013932 step 0 next 1296
2021-08-09 03:52:38.085088: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6400 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_t/_620 action_count 94195455013933 step 0 next 2303
2021-08-09 03:52:38.085105: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6500 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_f/_629 action_count 94195455013934 step 0 next 2357
2021-08-09 03:52:38.085124: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6600 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_t/_630 action_count 94195455013935 step 0 next 2273
2021-08-09 03:52:38.085148: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6700 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_f/_647 action_count 94195455013936 step 0 next 2365
2021-08-09 03:52:38.085170: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6800 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_t/_648 action_count 94195455013937 step 0 next 517
2021-08-09 03:52:38.085191: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6900 of size 256 by op gradient_tape/unet_depth3/encode1/batch_normalization_549/moments/scalar action_count 94195455013938 step 0 nex
t 2242
2021-08-09 03:52:38.085214: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6a00 of size 256 by op assert_less_equal_23/Assert/AssertGuard/pivot_f/_657 action_count 94195455013939 step 0 next 1406
2021-08-09 03:52:38.085234: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6b00 of size 256 by op assert_less_equal_23/Assert/AssertGuard/pivot_t/_658 action_count 94195455013940 step 0 next 1781
2021-08-09 03:52:38.085257: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6c00 of size 256 by op assert_greater_equal_24/Assert/AssertGuard/pivot_f/_675 action_count 94195455013941 step 0 next 1698
2021-08-09 03:52:38.085277: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6d00 of size 256 by op assert_greater_equal_24/Assert/AssertGuard/pivot_t/_676 action_count 94195455013942 step 0 next 2302
2021-08-09 03:52:38.085296: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6e00 of size 256 by op assert_less_equal_24/Assert/AssertGuard/pivot_f/_685 action_count 94195455013943 step 0 next 468
2021-08-09 03:52:38.085317: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd6f00 of size 256 by op assert_less_equal_24/Assert/AssertGuard/pivot_t/_686 action_count 94195455013944 step 0 next 2380
2021-08-09 03:52:38.085338: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7000 of size 256 by op gradient_tape/unet_depth3/encode1/batch_normalization_548/moments/scalar action_count 94195455013945 step 0 nex
t 2277
2021-08-09 03:52:38.085357: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7100 of size 256 by op assert_greater_equal_25/Assert/AssertGuard/pivot_f/_703 action_count 94195455013946 step 0 next 1686
2021-08-09 03:52:38.085379: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7200 of size 256 by op assert_greater_equal_25/Assert/AssertGuard/pivot_t/_704 action_count 94195455013947 step 0 next 2362
2021-08-09 03:52:38.085397: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7300 of size 256 by op assert_less_equal_25/Assert/AssertGuard/pivot_f/_713 action_count 94195455013948 step 0 next 2245
2021-08-09 03:52:38.085420: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7400 of size 256 by op assert_less_equal_25/Assert/AssertGuard/pivot_t/_714 action_count 94195455013949 step 0 next 2309
2021-08-09 03:52:38.085444: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7500 of size 256 by op assert_greater_equal_26/Assert/AssertGuard/pivot_f/_731 action_count 94195455013950 step 0 next 2461
2021-08-09 03:52:38.085465: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7600 of size 256 by op assert_greater_equal_26/Assert/AssertGuard/pivot_t/_732 action_count 94195455013951 step 0 next 2377
2021-08-09 03:52:38.085486: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7700 of size 256 by op assert_less_equal_26/Assert/AssertGuard/pivot_f/_741 action_count 94195455013952 step 0 next 2298
2021-08-09 03:52:38.085509: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7800 of size 256 by op assert_less_equal_26/Assert/AssertGuard/pivot_t/_742 action_count 94195455013953 step 0 next 2219
2021-08-09 03:52:38.085535: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7900 of size 256 by op gradient_tape/unet_depth3/encode0/batch_normalization_547/moments/scalar action_count 94195455013954 step 0 nex
t 1821
2021-08-09 03:52:38.085555: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7a00 of size 256 by op assert_greater_equal_27/Assert/AssertGuard/pivot_f/_759 action_count 94195455013955 step 0 next 1829
2021-08-09 03:52:38.085577: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7b00 of size 256 by op assert_greater_equal_27/Assert/AssertGuard/pivot_t/_760 action_count 94195455013956 step 0 next 2396
2021-08-09 03:52:38.085598: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7c00 of size 256 by op assert_less_equal_27/Assert/AssertGuard/pivot_f/_769 action_count 94195455013957 step 0 next 2255
2021-08-09 03:52:38.085620: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7d00 of size 256 by op assert_less_equal_27/Assert/AssertGuard/pivot_t/_770 action_count 94195455013958 step 0 next 573
2021-08-09 03:52:38.085641: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7e00 of size 256 by op Sub action_count 94195065726203 step 0 next 63
2021-08-09 03:52:38.085658: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd7f00 of size 256 by op Sub action_count 94195085385179 step 0 next 1215
2021-08-09 03:52:38.085675: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8000 of size 256 by op Sub action_count 94195085385180 step 0 next 631
2021-08-09 03:52:38.085696: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8100 of size 256 by op Sub action_count 94195094149558 step 0 next 1733
2021-08-09 03:52:38.085716: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8200 of size 256 by op Sub action_count 94195094149559 step 0 next 1648
2021-08-09 03:52:38.085736: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8300 of size 1792 by op Fill action_count 94195455013451 step 0 next 430
2021-08-09 03:52:38.085753: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8a00 of size 256 by op Sub action_count 94195065726216 step 0 next 425
2021-08-09 03:52:38.085770: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8b00 of size 512 by op Fill action_count 94195455013383 step 0 next 446
2021-08-09 03:52:38.085790: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8d00 of size 256 by op Sub action_count 94195065726217 step 0 next 1124
2021-08-09 03:52:38.085810: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8e00 of size 256 by op Sub action_count 94195085385138 step 0 next 1065
2021-08-09 03:52:38.085830: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd8f00 of size 256 by op Sub action_count 94195065726230 step 0 next 921
2021-08-09 03:52:38.085847: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9000 of size 256 by op Sub action_count 94195094149517 step 0 next 1149
2021-08-09 03:52:38.085868: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9100 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_f/_489 action_count 94195467563408 step 0 next 2477
2021-08-09 03:52:38.085889: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9200 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_t/_490 action_count 94195467563409 step 0 next 2456
2021-08-09 03:52:38.085909: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9300 of size 256 by op gradient_tape/unet_depth3/two_conv_center/batch_normalization_569/moments/scalar action_count 94195467563410 st
ep 0 next 1570
2021-08-09 03:52:38.085930: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9400 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_f/_507 action_count 94195467563411 step 0 next 2063
2021-08-09 03:52:38.085950: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9500 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_t/_508 action_count 94195467563412 step 0 next 2061
2021-08-09 03:52:38.085968: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9600 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_f/_517 action_count 94195467563413 step 0 next 244
2021-08-09 03:52:38.085989: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9700 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_t/_518 action_count 94195467563414 step 0 next 305
2021-08-09 03:52:38.086010: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9800 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_f/_535 action_count 94195467563415 step 0 next 317
2021-08-09 03:52:38.086027: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9900 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_t/_536 action_count 94195467563416 step 0 next 1228
2021-08-09 03:52:38.086045: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9a00 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_f/_545 action_count 94195467563417 step 0 next 746
2021-08-09 03:52:38.086063: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9b00 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_t/_546 action_count 94195467563418 step 0 next 979
2021-08-09 03:52:38.086084: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9c00 of size 256 by op gradient_tape/unet_depth3/encode2/batch_normalization_568/moments/scalar action_count 94195467563419 step 0 nex
t 1375
2021-08-09 03:52:38.086104: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9d00 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_f/_563 action_count 94195467563420 step 0 next 1207
2021-08-09 03:52:38.086125: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9e00 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_t/_564 action_count 94195467563421 step 0 next 1117
2021-08-09 03:52:38.086148: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfd9f00 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_f/_573 action_count 94195467563422 step 0 next 959
2021-08-09 03:52:38.086170: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda000 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_t/_574 action_count 94195467563423 step 0 next 1059
2021-08-09 03:52:38.086191: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda100 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_f/_591 action_count 94195467563424 step 0 next 246
2021-08-09 03:52:38.086213: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda200 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_t/_592 action_count 94195467563425 step 0 next 2589
2021-08-09 03:52:38.086235: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda300 of size 256 by op gradient_tape/unet_depth3/encode2/batch_normalization_567/moments/scalar action_count 94195467563426 step 0 nex
t 2465
2021-08-09 03:52:38.086257: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda400 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_f/_601 action_count 94195467563427 step 0 next 175
2021-08-09 03:52:38.086274: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda500 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_t/_602 action_count 94195467563428 step 0 next 1502
2021-08-09 03:52:38.086295: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda600 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_f/_619 action_count 94195467563429 step 0 next 1057
2021-08-09 03:52:38.086314: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda700 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_t/_620 action_count 94195467563430 step 0 next 173
2021-08-09 03:52:38.086338: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda800 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_f/_629 action_count 94195467563431 step 0 next 148
2021-08-09 03:52:38.086360: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfda900 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_t/_630 action_count 94195467563432 step 0 next 532
2021-08-09 03:52:38.086380: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdaa00 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_f/_647 action_count 94195467563433 step 0 next 2431
2021-08-09 03:52:38.086401: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdab00 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_t/_648 action_count 94195467563434 step 0 next 288
2021-08-09 03:52:38.086424: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdac00 of size 256 by op gradient_tape/unet_depth3/encode1/batch_normalization_566/moments/scalar action_count 94195467563435 step 0 nex
t 1144
2021-08-09 03:52:38.086444: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdad00 of size 256 by op assert_less_equal_23/Assert/AssertGuard/pivot_f/_657 action_count 94195467563436 step 0 next 1242
2021-08-09 03:52:38.086467: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdae00 of size 256 by op assert_less_equal_23/Assert/AssertGuard/pivot_t/_658 action_count 94195467563437 step 0 next 774
2021-08-09 03:52:38.086489: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdaf00 of size 256 by op assert_greater_equal_24/Assert/AssertGuard/pivot_f/_675 action_count 94195467563438 step 0 next 593
2021-08-09 03:52:38.086507: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb000 of size 256 by op assert_greater_equal_24/Assert/AssertGuard/pivot_t/_676 action_count 94195467563439 step 0 next 1856
2021-08-09 03:52:38.086533: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb100 of size 256 by op assert_less_equal_24/Assert/AssertGuard/pivot_f/_685 action_count 94195467563440 step 0 next 1759
2021-08-09 03:52:38.086551: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb200 of size 256 by op assert_less_equal_24/Assert/AssertGuard/pivot_t/_686 action_count 94195467563441 step 0 next 86
2021-08-09 03:52:38.086574: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb300 of size 256 by op gradient_tape/unet_depth3/encode1/batch_normalization_565/moments/scalar action_count 94195467563442 step 0 nex
t 295
2021-08-09 03:52:38.086595: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb400 of size 256 by op assert_greater_equal_25/Assert/AssertGuard/pivot_f/_703 action_count 94195467563443 step 0 next 2052
2021-08-09 03:52:38.086615: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb500 of size 256 by op assert_greater_equal_25/Assert/AssertGuard/pivot_t/_704 action_count 94195467563444 step 0 next 1063
2021-08-09 03:52:38.086633: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb600 of size 256 by op assert_less_equal_25/Assert/AssertGuard/pivot_f/_713 action_count 94195467563445 step 0 next 2449
2021-08-09 03:52:38.086653: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb700 of size 256 by op assert_less_equal_25/Assert/AssertGuard/pivot_t/_714 action_count 94195467563446 step 0 next 2414
2021-08-09 03:52:38.086671: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb800 of size 256 by op assert_greater_equal_26/Assert/AssertGuard/pivot_f/_731 action_count 94195467563447 step 0 next 1213
2021-08-09 03:52:38.086691: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdb900 of size 256 by op assert_greater_equal_26/Assert/AssertGuard/pivot_t/_732 action_count 94195467563448 step 0 next 2266
2021-08-09 03:52:38.086712: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdba00 of size 256 by op assert_less_equal_26/Assert/AssertGuard/pivot_f/_741 action_count 94195467563449 step 0 next 866
2021-08-09 03:52:38.086732: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdbb00 of size 256 by op assert_less_equal_26/Assert/AssertGuard/pivot_t/_742 action_count 94195467563450 step 0 next 42
2021-08-09 03:52:38.086753: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdbc00 of size 256 by op gradient_tape/unet_depth3/encode0/batch_normalization_564/moments/scalar action_count 94195467563451 step 0 nex
t 935
2021-08-09 03:52:38.086775: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdbd00 of size 256 by op assert_greater_equal_27/Assert/AssertGuard/pivot_f/_759 action_count 94195467563452 step 0 next 929
2021-08-09 03:52:38.086792: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdbe00 of size 256 by op assert_greater_equal_27/Assert/AssertGuard/pivot_t/_760 action_count 94195467563453 step 0 next 533
2021-08-09 03:52:38.086814: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdbf00 of size 256 by op assert_less_equal_27/Assert/AssertGuard/pivot_f/_769 action_count 94195467563454 step 0 next 763
2021-08-09 03:52:38.086838: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc000 of size 256 by op assert_less_equal_27/Assert/AssertGuard/pivot_t/_770 action_count 94195467563455 step 0 next 1804
2021-08-09 03:52:38.086859: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc100 of size 256 by op assert_greater_equal_28/Assert/AssertGuard/pivot_f/_787 action_count 94195467563456 step 0 next 1857
2021-08-09 03:52:38.086880: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc200 of size 256 by op assert_greater_equal_28/Assert/AssertGuard/pivot_t/_788 action_count 94195467563457 step 0 next 1084
2021-08-09 03:52:38.086902: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc300 of size 256 by op assert_less_equal_28/Assert/AssertGuard/pivot_f/_797 action_count 94195467563458 step 0 next 1122
2021-08-09 03:52:38.086920: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc400 of size 256 by op assert_less_equal_28/Assert/AssertGuard/pivot_t/_798 action_count 94195467563459 step 0 next 2180
2021-08-09 03:52:38.086941: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc500 of size 256 by op gradient_tape/unet_depth3/encode0/batch_normalization_563/moments/scalar action_count 94195467563460 step 0 nex
t 1080
2021-08-09 03:52:38.086962: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc600 of size 256 by op assert_greater_equal_29/Assert/AssertGuard/pivot_f/_815 action_count 94195467563461 step 0 next 89
2021-08-09 03:52:38.086979: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc700 of size 256 by op assert_greater_equal_29/Assert/AssertGuard/pivot_t/_816 action_count 94195467563462 step 0 next 2442
2021-08-09 03:52:38.087001: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc800 of size 256 by op Adam/Adam/Const action_count 94195467563463 step 0 next 1158
2021-08-09 03:52:38.087023: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdc900 of size 256 by op assert_less_equal_29/Assert/AssertGuard/pivot_f/_825 action_count 94195467563464 step 0 next 580
2021-08-09 03:52:38.087041: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdca00 of size 256 by op assert_less_equal_29/Assert/AssertGuard/pivot_t/_826 action_count 94195467563465 step 0 next 71
2021-08-09 03:52:38.087058: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdcb00 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_f/_843 action_count 94195467563466 step 0 next 993
2021-08-09 03:52:38.087076: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdcc00 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_t/_844 action_count 94195467563467 step 0 next 1831
2021-08-09 03:52:38.087098: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdcd00 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_f/_853 action_count 94195467563468 step 0 next 1884
2021-08-09 03:52:38.087118: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdce00 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_t/_854 action_count 94195467563469 step 0 next 1294
2021-08-09 03:52:38.087139: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdcf00 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_f/_871 action_count 94195467563470 step 0 next 1901
2021-08-09 03:52:38.087159: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd000 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_t/_872 action_count 94195467563471 step 0 next 331
2021-08-09 03:52:38.087178: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd100 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_f/_881 action_count 94195467563472 step 0 next 1340
2021-08-09 03:52:38.087199: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd200 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_t/_882 action_count 94195467563473 step 0 next 208
2021-08-09 03:52:38.087217: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd300 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_f/_899 action_count 94195467563474 step 0 next 2434
2021-08-09 03:52:38.087234: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd400 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_t/_900 action_count 94195467563475 step 0 next 287
2021-08-09 03:52:38.087251: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd500 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_f/_909 action_count 94195467563476 step 0 next 745
2021-08-09 03:52:38.087273: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd600 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_t/_910 action_count 94195467563477 step 0 next 206
2021-08-09 03:52:38.087297: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd700 of size 256 by op assert_greater_equal_5/Assert/AssertGuard/pivot_f/_143 action_count 94195468133277 step 0 next 455
2021-08-09 03:52:38.087314: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd800 of size 256 by op assert_greater_equal_5/Assert/AssertGuard/pivot_t/_144 action_count 94195468133278 step 0 next 400
2021-08-09 03:52:38.087336: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdd900 of size 256 by op assert_less_equal_5/Assert/AssertGuard/pivot_f/_153 action_count 94195468133279 step 0 next 1081
2021-08-09 03:52:38.087357: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdda00 of size 256 by op assert_less_equal_5/Assert/AssertGuard/pivot_t/_154 action_count 94195468133280 step 0 next 2184
2021-08-09 03:52:38.087380: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfddb00 of size 256 by op assert_greater_equal_6/Assert/AssertGuard/pivot_f/_171 action_count 94195468133281 step 0 next 2247
2021-08-09 03:52:38.087401: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfddc00 of size 256 by op assert_greater_equal_6/Assert/AssertGuard/pivot_t/_172 action_count 94195468133282 step 0 next 1551
2021-08-09 03:52:38.087422: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfddd00 of size 256 by op assert_less_equal_6/Assert/AssertGuard/pivot_f/_181 action_count 94195468133283 step 0 next 2030
2021-08-09 03:52:38.087443: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdde00 of size 256 by op assert_less_equal_6/Assert/AssertGuard/pivot_t/_182 action_count 94195468133284 step 0 next 1665
2021-08-09 03:52:38.087460: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfddf00 of size 256 by op assert_greater_equal_7/Assert/AssertGuard/pivot_f/_199 action_count 94195468133285 step 0 next 1352
2021-08-09 03:52:38.087483: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde000 of size 256 by op assert_greater_equal_7/Assert/AssertGuard/pivot_t/_200 action_count 94195468133286 step 0 next 714
2021-08-09 03:52:38.087506: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde100 of size 256 by op assert_less_equal_7/Assert/AssertGuard/pivot_f/_209 action_count 94195468133287 step 0 next 149
2021-08-09 03:52:38.087528: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde200 of size 256 by op assert_less_equal_7/Assert/AssertGuard/pivot_t/_210 action_count 94195468133288 step 0 next 2132
2021-08-09 03:52:38.087546: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde300 of size 256 by op assert_greater_equal_8/Assert/AssertGuard/pivot_f/_227 action_count 94195468133289 step 0 next 374
2021-08-09 03:52:38.087564: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde400 of size 256 by op assert_greater_equal_8/Assert/AssertGuard/pivot_t/_228 action_count 94195468133290 step 0 next 1025
2021-08-09 03:52:38.087588: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde500 of size 256 by op assert_less_equal_8/Assert/AssertGuard/pivot_f/_237 action_count 94195468133291 step 0 next 2645
2021-08-09 03:52:38.087609: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde600 of size 256 by op assert_less_equal_8/Assert/AssertGuard/pivot_t/_238 action_count 94195468133292 step 0 next 2179
2021-08-09 03:52:38.087630: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde700 of size 256 by op assert_greater_equal_9/Assert/AssertGuard/pivot_f/_255 action_count 94195468133293 step 0 next 800
2021-08-09 03:52:38.087649: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde800 of size 256 by op assert_greater_equal_9/Assert/AssertGuard/pivot_t/_256 action_count 94195468133294 step 0 next 2136
2021-08-09 03:52:38.087672: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfde900 of size 256 by op assert_less_equal_9/Assert/AssertGuard/pivot_f/_265 action_count 94195468133295 step 0 next 1320
2021-08-09 03:52:38.087695: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdea00 of size 256 by op assert_less_equal_9/Assert/AssertGuard/pivot_t/_266 action_count 94195468133296 step 0 next 1892
2021-08-09 03:52:38.087717: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdeb00 of size 256 by op assert_greater_equal_10/Assert/AssertGuard/pivot_f/_283 action_count 94195468133297 step 0 next 1601
2021-08-09 03:52:38.087736: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdec00 of size 256 by op assert_greater_equal_10/Assert/AssertGuard/pivot_t/_284 action_count 94195468133298 step 0 next 196
2021-08-09 03:52:38.087754: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfded00 of size 256 by op assert_less_equal_10/Assert/AssertGuard/pivot_f/_293 action_count 94195468133299 step 0 next 591
2021-08-09 03:52:38.087776: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdee00 of size 256 by op assert_less_equal_10/Assert/AssertGuard/pivot_t/_294 action_count 94195468133300 step 0 next 396
2021-08-09 03:52:38.087799: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdef00 of size 256 by op assert_greater_equal_11/Assert/AssertGuard/pivot_f/_311 action_count 94195468133301 step 0 next 1814
2021-08-09 03:52:38.087820: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf000 of size 256 by op assert_greater_equal_11/Assert/AssertGuard/pivot_t/_312 action_count 94195468133302 step 0 next 1236
2021-08-09 03:52:38.087843: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf100 of size 256 by op assert_less_equal_11/Assert/AssertGuard/pivot_f/_321 action_count 94195468133303 step 0 next 2579
2021-08-09 03:52:38.087863: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf200 of size 256 by op assert_less_equal_11/Assert/AssertGuard/pivot_t/_322 action_count 94195468133304 step 0 next 2426
2021-08-09 03:52:38.087884: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf300 of size 256 by op assert_greater_equal_12/Assert/AssertGuard/pivot_f/_339 action_count 94195468133305 step 0 next 1668
2021-08-09 03:52:38.087904: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf400 of size 256 by op assert_greater_equal_12/Assert/AssertGuard/pivot_t/_340 action_count 94195468133306 step 0 next 1310
2021-08-09 03:52:38.087928: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf500 of size 256 by op assert_less_equal_12/Assert/AssertGuard/pivot_f/_349 action_count 94195468133307 step 0 next 479
2021-08-09 03:52:38.087945: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf600 of size 256 by op assert_less_equal_12/Assert/AssertGuard/pivot_t/_350 action_count 94195468133308 step 0 next 19
2021-08-09 03:52:38.087968: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf700 of size 256 by op assert_greater_equal_13/Assert/AssertGuard/pivot_f/_367 action_count 94195468133309 step 0 next 2079
2021-08-09 03:52:38.087988: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf800 of size 256 by op assert_greater_equal_13/Assert/AssertGuard/pivot_t/_368 action_count 94195468133310 step 0 next 585
2021-08-09 03:52:38.088011: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdf900 of size 256 by op assert_less_equal_13/Assert/AssertGuard/pivot_f/_377 action_count 94195468133311 step 0 next 1988
2021-08-09 03:52:38.088032: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdfa00 of size 256 by op assert_less_equal_13/Assert/AssertGuard/pivot_t/_378 action_count 94195468133312 step 0 next 1580
2021-08-09 03:52:38.088054: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdfb00 of size 256 by op assert_greater_equal_14/Assert/AssertGuard/pivot_f/_395 action_count 94195468133313 step 0 next 277
2021-08-09 03:52:38.088072: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdfc00 of size 256 by op assert_greater_equal_14/Assert/AssertGuard/pivot_t/_396 action_count 94195468133314 step 0 next 2467
2021-08-09 03:52:38.088089: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdfd00 of size 256 by op assert_less_equal_14/Assert/AssertGuard/pivot_f/_405 action_count 94195468133315 step 0 next 870
2021-08-09 03:52:38.088109: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdfe00 of size 256 by op assert_less_equal_14/Assert/AssertGuard/pivot_t/_406 action_count 94195468133316 step 0 next 702
2021-08-09 03:52:38.088130: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfdff00 of size 256 by op assert_greater_equal_15/Assert/AssertGuard/pivot_f/_423 action_count 94195468133317 step 0 next 2279
2021-08-09 03:52:38.088152: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0000 of size 256 by op assert_greater_equal_15/Assert/AssertGuard/pivot_t/_424 action_count 94195468133318 step 0 next 452
2021-08-09 03:52:38.088173: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0100 of size 256 by op assert_less_equal_15/Assert/AssertGuard/pivot_f/_433 action_count 94195468133319 step 0 next 1777
2021-08-09 03:52:38.088194: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0200 of size 256 by op assert_less_equal_15/Assert/AssertGuard/pivot_t/_434 action_count 94195468133320 step 0 next 974
2021-08-09 03:52:38.088213: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0300 of size 256 by op assert_greater_equal_16/Assert/AssertGuard/pivot_f/_451 action_count 94195468133321 step 0 next 831
2021-08-09 03:52:38.088235: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0400 of size 256 by op assert_greater_equal_16/Assert/AssertGuard/pivot_t/_452 action_count 94195468133322 step 0 next 302
2021-08-09 03:52:38.088252: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0500 of size 256 by op assert_less_equal_16/Assert/AssertGuard/pivot_f/_461 action_count 94195468133323 step 0 next 1034
2021-08-09 03:52:38.088270: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0600 of size 256 by op assert_less_equal_16/Assert/AssertGuard/pivot_t/_462 action_count 94195468133324 step 0 next 1976
2021-08-09 03:52:38.088293: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0700 of size 256 by op assert_greater_equal_17/Assert/AssertGuard/pivot_f/_479 action_count 94195468133325 step 0 next 660
2021-08-09 03:52:38.088314: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0800 of size 256 by op assert_greater_equal_17/Assert/AssertGuard/pivot_t/_480 action_count 94195468133326 step 0 next 618
2021-08-09 03:52:38.088337: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0900 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_f/_489 action_count 94195468133327 step 0 next 1226
2021-08-09 03:52:38.088360: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0a00 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_t/_490 action_count 94195468133328 step 0 next 688
2021-08-09 03:52:38.088383: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0b00 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_f/_507 action_count 94195468133329 step 0 next 1711
2021-08-09 03:52:38.088405: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0c00 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_t/_508 action_count 94195468133330 step 0 next 1005
2021-08-09 03:52:38.088425: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0d00 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_f/_517 action_count 94195468133331 step 0 next 77
2021-08-09 03:52:38.088445: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0e00 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_t/_518 action_count 94195468133332 step 0 next 639
2021-08-09 03:52:38.088466: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe0f00 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_f/_535 action_count 94195468133333 step 0 next 1977
2021-08-09 03:52:38.088484: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1000 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_t/_536 action_count 94195468133334 step 0 next 190
2021-08-09 03:52:38.088505: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1100 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_f/_545 action_count 94195468133335 step 0 next 476
2021-08-09 03:52:38.088539: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1200 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_t/_546 action_count 94195468133336 step 0 next 923
2021-08-09 03:52:38.088580: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1300 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_f/_563 action_count 94195468133337 step 0 next 624
2021-08-09 03:52:38.088613: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1400 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_t/_564 action_count 94195468133338 step 0 next 1916
2021-08-09 03:52:38.088638: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1500 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_f/_573 action_count 94195468133339 step 0 next 1308
2021-08-09 03:52:38.088661: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1600 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_t/_574 action_count 94195468133340 step 0 next 251
2021-08-09 03:52:38.088682: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1700 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_f/_591 action_count 94195468133341 step 0 next 2471
2021-08-09 03:52:38.088703: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1800 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_t/_592 action_count 94195468133342 step 0 next 35
2021-08-09 03:52:38.088725: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1900 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_f/_601 action_count 94195468133343 step 0 next 2394
2021-08-09 03:52:38.088743: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1a00 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_t/_602 action_count 94195468133344 step 0 next 880
2021-08-09 03:52:38.088765: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1b00 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_f/_619 action_count 94195468133345 step 0 next 977
2021-08-09 03:52:38.088787: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1c00 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_t/_620 action_count 94195468133346 step 0 next 682
2021-08-09 03:52:38.088809: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1d00 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_f/_629 action_count 94195468133347 step 0 next 1531
2021-08-09 03:52:38.088827: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1e00 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_t/_630 action_count 94195468133348 step 0 next 904
2021-08-09 03:52:38.088850: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe1f00 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_f/_647 action_count 94195468133349 step 0 next 1524
2021-08-09 03:52:38.088872: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2000 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_t/_648 action_count 94195468133350 step 0 next 1713
2021-08-09 03:52:38.088893: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2100 of size 256 by op assert_less_equal_23/Assert/AssertGuard/pivot_f/_657 action_count 94195468133351 step 0 next 829
2021-08-09 03:52:38.088913: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2200 of size 256 by op assert_less_equal_23/Assert/AssertGuard/pivot_t/_658 action_count 94195468133352 step 0 next 1533
2021-08-09 03:52:38.088932: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2300 of size 256 by op assert_greater_equal_24/Assert/AssertGuard/pivot_f/_675 action_count 94195468133353 step 0 next 1279
2021-08-09 03:52:38.088954: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2400 of size 256 by op assert_greater_equal_24/Assert/AssertGuard/pivot_t/_676 action_count 94195468133354 step 0 next 1639
2021-08-09 03:52:38.088977: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2500 of size 256 by op assert_less_equal_24/Assert/AssertGuard/pivot_f/_685 action_count 94195468133355 step 0 next 179
2021-08-09 03:52:38.088998: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2600 of size 256 by op assert_less_equal_24/Assert/AssertGuard/pivot_t/_686 action_count 94195468133356 step 0 next 1984
2021-08-09 03:52:38.089019: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2700 of size 256 by op assert_greater_equal_25/Assert/AssertGuard/pivot_f/_703 action_count 94195468133357 step 0 next 1918
2021-08-09 03:52:38.089040: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2800 of size 256 by op assert_greater_equal_25/Assert/AssertGuard/pivot_t/_704 action_count 94195468133358 step 0 next 329
2021-08-09 03:52:38.089059: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2900 of size 256 by op assert_less_equal_25/Assert/AssertGuard/pivot_f/_713 action_count 94195468133359 step 0 next 2060
2021-08-09 03:52:38.089083: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2a00 of size 256 by op assert_less_equal_25/Assert/AssertGuard/pivot_t/_714 action_count 94195468133360 step 0 next 1820
2021-08-09 03:52:38.089100: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2b00 of size 256 by op assert_greater_equal_26/Assert/AssertGuard/pivot_f/_731 action_count 94195468133361 step 0 next 1874
2021-08-09 03:52:38.089122: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2c00 of size 256 by op assert_greater_equal_26/Assert/AssertGuard/pivot_t/_732 action_count 94195468133362 step 0 next 1408
2021-08-09 03:52:38.089154: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2d00 of size 256 by op assert_less_equal_26/Assert/AssertGuard/pivot_f/_741 action_count 94195468133363 step 0 next 584
2021-08-09 03:52:38.089170: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2e00 of size 256 by op assert_less_equal_26/Assert/AssertGuard/pivot_t/_742 action_count 94195468133364 step 0 next 1372
2021-08-09 03:52:38.089192: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe2f00 of size 256 by op assert_greater_equal_27/Assert/AssertGuard/pivot_f/_759 action_count 94195468133365 step 0 next 43
2021-08-09 03:52:38.089213: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3000 of size 256 by op assert_greater_equal_27/Assert/AssertGuard/pivot_t/_760 action_count 94195468133366 step 0 next 1041
2021-08-09 03:52:38.089233: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3100 of size 256 by op assert_less_equal_27/Assert/AssertGuard/pivot_f/_769 action_count 94195468133367 step 0 next 1261
2021-08-09 03:52:38.089254: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3200 of size 256 by op assert_less_equal_27/Assert/AssertGuard/pivot_t/_770 action_count 94195468133368 step 0 next 1939
2021-08-09 03:52:38.089275: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3300 of size 256 by op assert_greater_equal_28/Assert/AssertGuard/pivot_f/_787 action_count 94195468133369 step 0 next 576
2021-08-09 03:52:38.089295: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3400 of size 256 by op assert_greater_equal_28/Assert/AssertGuard/pivot_t/_788 action_count 94195468133370 step 0 next 1836
2021-08-09 03:52:38.089315: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3500 of size 256 by op assert_less_equal_28/Assert/AssertGuard/pivot_f/_797 action_count 94195468133371 step 0 next 1749
2021-08-09 03:52:38.089335: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3600 of size 256 by op assert_less_equal_28/Assert/AssertGuard/pivot_t/_798 action_count 94195468133372 step 0 next 36
2021-08-09 03:52:38.089356: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3700 of size 256 by op assert_greater_equal_29/Assert/AssertGuard/pivot_f/_815 action_count 94195468133373 step 0 next 1938
2021-08-09 03:52:38.089377: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3800 of size 256 by op assert_greater_equal_29/Assert/AssertGuard/pivot_t/_816 action_count 94195468133374 step 0 next 2130
2021-08-09 03:52:38.089396: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3900 of size 256 by op assert_less_equal_29/Assert/AssertGuard/pivot_f/_825 action_count 94195468133375 step 0 next 2473
2021-08-09 03:52:38.089416: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3a00 of size 256 by op assert_less_equal_29/Assert/AssertGuard/pivot_t/_826 action_count 94195468133376 step 0 next 348
2021-08-09 03:52:38.089436: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3b00 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_f/_843 action_count 94195468133377 step 0 next 1911
2021-08-09 03:52:38.089455: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3c00 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_t/_844 action_count 94195468133378 step 0 next 555
2021-08-09 03:52:38.089476: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3d00 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_f/_853 action_count 94195468133379 step 0 next 2120
2021-08-09 03:52:38.089497: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3e00 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_t/_854 action_count 94195468133380 step 0 next 543
2021-08-09 03:52:38.089523: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe3f00 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_f/_871 action_count 94195468133381 step 0 next 2594
2021-08-09 03:52:38.089546: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4000 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_t/_872 action_count 94195468133382 step 0 next 625
2021-08-09 03:52:38.089565: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4100 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_f/_881 action_count 94195468133383 step 0 next 2083
2021-08-09 03:52:38.089585: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4200 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_t/_882 action_count 94195468133384 step 0 next 239
2021-08-09 03:52:38.089604: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4300 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_f/_899 action_count 94195468133385 step 0 next 319
2021-08-09 03:52:38.089624: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4400 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_t/_900 action_count 94195468133386 step 0 next 327
2021-08-09 03:52:38.089646: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4500 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_f/_909 action_count 94195468133387 step 0 next 93
2021-08-09 03:52:38.089666: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4600 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_t/_910 action_count 94195468133388 step 0 next 1947
2021-08-09 03:52:38.089685: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4700 of size 256 by op Sub action_count 94195480110842 step 0 next 540
2021-08-09 03:52:38.089704: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4800 of size 256 by op AssignVariableOp action_count 94195480053210 step 0 next 316
2021-08-09 03:52:38.089724: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4900 of size 256 by op AssignVariableOp action_count 94195480053212 step 0 next 2347
2021-08-09 03:52:38.089743: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4a00 of size 512 by op Fill action_count 94195480110824 step 0 next 2410
2021-08-09 03:52:38.089762: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4c00 of size 256 by op AssignVariableOp action_count 94195480053246 step 0 next 1466
2021-08-09 03:52:38.089782: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4d00 of size 256 by op AssignVariableOp action_count 94195480053250 step 0 next 2149
2021-08-09 03:52:38.089802: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4e00 of size 256 by op AssignVariableOp action_count 94195480053252 step 0 next 356
2021-08-09 03:52:38.089821: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe4f00 of size 256 by op AssignVariableOp action_count 94195480053254 step 0 next 2000
2021-08-09 03:52:38.089840: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5000 of size 256 by op AssignVariableOp action_count 94195480053256 step 0 next 2212
2021-08-09 03:52:38.089859: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5100 of size 512 by op AssignVariableOp action_count 94195480053258 step 0 next 872
2021-08-09 03:52:38.089879: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5300 of size 512 by op AssignVariableOp action_count 94195480053262 step 0 next 23
2021-08-09 03:52:38.089899: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5500 of size 256 by op AssignVariableOp action_count 94195480053266 step 0 next 518
2021-08-09 03:52:38.089918: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5600 of size 256 by op AssignVariableOp action_count 94195480053268 step 0 next 1096
2021-08-09 03:52:38.089937: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5700 of size 256 by op AssignVariableOp action_count 94195480053270 step 0 next 2493
2021-08-09 03:52:38.089956: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5800 of size 256 by op AssignVariableOp action_count 94195480053272 step 0 next 2173
2021-08-09 03:52:38.089975: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5900 of size 512 by op AssignVariableOp action_count 94195480053260 step 0 next 1908
2021-08-09 03:52:38.089995: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5b00 of size 512 by op AssignVariableOp action_count 94195480053264 step 0 next 1606
2021-08-09 03:52:38.090013: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5d00 of size 512 by op Fill action_count 94195480110825 step 0 next 1046
2021-08-09 03:52:38.090031: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe5f00 of size 256 by op AssignAddVariableOp_3 action_count 94195480109333 step 17368542405085625385 next 811
2021-08-09 03:52:38.090050: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe6000 of size 256 by op Sub action_count 94195480110843 step 0 next 854
2021-08-09 03:52:38.090070: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe6100 of size 256 by op AssignAddVariableOp_13 action_count 94195480109431 step 17368542405085625385 next 2268
2021-08-09 03:52:38.090091: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe6200 of size 256 by op AssignAddVariableOp_19 action_count 94195480109375 step 17368542405085625385 next 1085
2021-08-09 03:52:38.090112: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe6300 of size 256 by op AssignAddVariableOp_21 action_count 94195480109435 step 17368542405085625385 next 1013
2021-08-09 03:52:38.090131: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe6400 of size 256 by op Sub action_count 94195094149710 step 0 next 1794
2021-08-09 03:52:38.090150: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe6500 of size 256 by op Sub action_count 94195094149711 step 0 next 1075
2021-08-09 03:52:38.090169: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe6600 of size 1792 by op Fill action_count 94195455013684 step 0 next 1463
2021-08-09 03:52:38.090188: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe6d00 of size 1792 by op Fill action_count 94195455013686 step 0 next 401
2021-08-09 03:52:38.090207: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe7400 of size 1792 by op Fill action_count 94195455013687 step 0 next 1373
2021-08-09 03:52:38.090226: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe7b00 of size 2304 by op Fill action_count 94195455013688 step 0 next 1595
2021-08-09 03:52:38.090244: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe8400 of size 256 by op Sub action_count 94195094149724 step 0 next 1589
2021-08-09 03:52:38.090263: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe8500 of size 256 by op Sub action_count 94195094149725 step 0 next 1735
2021-08-09 03:52:38.090282: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe8600 of size 2048 by op Fill action_count 94195467563205 step 0 next 1792
2021-08-09 03:52:38.090302: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe8e00 of size 1792 by op Fill action_count 94195467563207 step 0 next 528
2021-08-09 03:52:38.090323: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe9500 of size 1792 by op Fill action_count 94195467563208 step 0 next 711
2021-08-09 03:52:38.090341: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfe9c00 of size 1792 by op Fill action_count 94195467563209 step 0 next 437
2021-08-09 03:52:38.090362: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfea300 of size 1792 by op Fill action_count 94195467563211 step 0 next 2070
2021-08-09 03:52:38.090381: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfeaa00 of size 1792 by op Fill action_count 94195467563212 step 0 next 2169
2021-08-09 03:52:38.090400: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfeb100 of size 1792 by op Fill action_count 94195467563213 step 0 next 1646
2021-08-09 03:52:38.090419: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfeb800 of size 1792 by op Fill action_count 94195467563215 step 0 next 1692
2021-08-09 03:52:38.090438: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfebf00 of size 1792 by op Fill action_count 94195467563216 step 0 next 1654
2021-08-09 03:52:38.090457: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfec600 of size 1792 by op Fill action_count 94195467563217 step 0 next 87
2021-08-09 03:52:38.090476: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfecd00 of size 1024 by op Fill action_count 94195467563219 step 0 next 2186
2021-08-09 03:52:38.090495: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfed100 of size 1024 by op Fill action_count 94195467563220 step 0 next 1407
2021-08-09 03:52:38.090513: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfed500 of size 1024 by op Fill action_count 94195467563221 step 0 next 937
2021-08-09 03:52:38.090544: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfed900 of size 1024 by op Fill action_count 94195467563223 step 0 next 1812
2021-08-09 03:52:38.090561: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfedd00 of size 1024 by op Fill action_count 94195467563224 step 0 next 2446
2021-08-09 03:52:38.090576: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfee100 of size 1024 by op Fill action_count 94195467563225 step 0 next 1989
2021-08-09 03:52:38.090595: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfee500 of size 1024 by op Fill action_count 94195467563227 step 0 next 2192
2021-08-09 03:52:38.090614: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfee900 of size 1024 by op Fill action_count 94195467563228 step 0 next 473
2021-08-09 03:52:38.090635: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfeed00 of size 1024 by op Fill action_count 94195467563229 step 0 next 1201
2021-08-09 03:52:38.090653: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfef100 of size 1024 by op Fill action_count 94195467563230 step 0 next 2573
2021-08-09 03:52:38.090672: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfef500 of size 256 by op Fill action_count 94195467563231 step 0 next 508
2021-08-09 03:52:38.090691: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfef600 of size 1536 by op Fill action_count 94195467563232 step 0 next 684
2021-08-09 03:52:38.090710: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfefc00 of size 512 by op Fill action_count 94195467563233 step 0 next 579
2021-08-09 03:52:38.090730: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfefe00 of size 512 by op Fill action_count 94195467563234 step 0 next 1756
2021-08-09 03:52:38.090749: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff0000 of size 512 by op Fill action_count 94195467563235 step 0 next 1766
2021-08-09 03:52:38.090768: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff0200 of size 512 by op Fill action_count 94195467563237 step 0 next 648
2021-08-09 03:52:38.090787: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff0400 of size 512 by op Fill action_count 94195467563238 step 0 next 2663
2021-08-09 03:52:38.090806: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff0600 of size 512 by op Fill action_count 94195467563239 step 0 next 527
2021-08-09 03:52:38.090825: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff0800 of size 1024 by op Fill action_count 94195467563241 step 0 next 267
2021-08-09 03:52:38.090844: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff0c00 of size 1024 by op Fill action_count 94195467563242 step 0 next 553
2021-08-09 03:52:38.090860: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff1000 of size 1024 by op Fill action_count 94195467563243 step 0 next 1850
2021-08-09 03:52:38.090880: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff1400 of size 1024 by op Fill action_count 94195467563245 step 0 next 235
2021-08-09 03:52:38.090899: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff1800 of size 1024 by op Fill action_count 94195467563246 step 0 next 550
2021-08-09 03:52:38.090918: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff1c00 of size 1024 by op Fill action_count 94195467563247 step 0 next 323
2021-08-09 03:52:38.090937: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff2000 of size 1792 by op Fill action_count 94195467563249 step 0 next 723
2021-08-09 03:52:38.090956: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff2700 of size 1792 by op Fill action_count 94195467563250 step 0 next 152
2021-08-09 03:52:38.090975: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff2e00 of size 2304 by op Fill action_count 94195467563251 step 0 next 1742
2021-08-09 03:52:38.090992: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff3700 of size 256 by op Sub action_count 94195094149762 step 0 next 1696
2021-08-09 03:52:38.091008: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff3800 of size 256 by op Sub action_count 94195094149763 step 0 next 1624
2021-08-09 03:52:38.091027: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff3900 of size 3584 by op Fill action_count 94195455013762 step 0 next 495
2021-08-09 03:52:38.091047: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff4700 of size 6656 by op Fill action_count 94195455013764 step 0 next 464
2021-08-09 03:52:38.091063: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6100 of size 256 by op Sub action_count 94195094149800 step 0 next 1592
2021-08-09 03:52:38.091079: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6200 of size 256 by op Sub action_count 94195094149801 step 0 next 1619
2021-08-09 03:52:38.091095: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6300 of size 1024 by op Fill action_count 94195455013599 step 0 next 1110
2021-08-09 03:52:38.091115: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6700 of size 1024 by op Fill action_count 94195455013600 step 0 next 1927
2021-08-09 03:52:38.091134: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6b00 of size 256 by op Fill action_count 94195455013610 step 0 next 2528
2021-08-09 03:52:38.091153: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6c00 of size 256 by op Sub action_count 94195480110856 step 0 next 842
2021-08-09 03:52:38.091175: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6d00 of size 256 by op AssignAddVariableOp_10 action_count 94195467561440 step 17436438550591945738 next 2050
2021-08-09 03:52:38.091192: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6e00 of size 256 by op Sub action_count 94195480110857 step 0 next 1259
2021-08-09 03:52:38.091208: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff6f00 of size 256 by op Sub action_count 94195455013602 step 0 next 2800
2021-08-09 03:52:38.091224: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7000 of size 256 by op Sub action_count 94195455013603 step 0 next 2780
2021-08-09 03:52:38.091239: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7100 of size 256 by op AssignVariableOp action_count 94195467505275 step 0 next 44
2021-08-09 03:52:38.091260: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7200 of size 256 by op AssignVariableOp action_count 94195467505273 step 0 next 823
2021-08-09 03:52:38.091279: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7300 of size 256 by op AssignVariableOp action_count 94195467505277 step 0 next 819
2021-08-09 03:52:38.091298: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7400 of size 256 by op Sub action_count 94195480110870 step 0 next 2018
2021-08-09 03:52:38.091318: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7500 of size 256 by op Const action_count 94195455640678 step 0 next 3047
2021-08-09 03:52:38.091339: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7600 of size 1536 by op Add action_count 94195455013607 step 0 next 1651
2021-08-09 03:52:38.091355: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7c00 of size 256 by op Sub action_count 94195094149838 step 0 next 1743
2021-08-09 03:52:38.091371: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7d00 of size 256 by op Sub action_count 94195094149839 step 0 next 1559
2021-08-09 03:52:38.091387: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7e00 of size 256 by op Sub action_count 94195299052790 step 0 next 409
2021-08-09 03:52:38.091403: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff7f00 of size 2560 by op Fill action_count 94195455013495 step 0 next 2405
2021-08-09 03:52:38.091419: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff8900 of size 256 by op Sub action_count 94195299052803 step 0 next 343
2021-08-09 03:52:38.091435: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff8a00 of size 256 by op Sub action_count 94195455013498 step 0 next 2753
2021-08-09 03:52:38.091454: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff8b00 of size 256 by op Sub action_count 94195455013499 step 0 next 2925
2021-08-09 03:52:38.091470: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff8c00 of size 2304 by op Fill action_count 94195455013506 step 0 next 1906
2021-08-09 03:52:38.091487: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff9500 of size 256 by op Sub action_count 94195299052804 step 0 next 264
2021-08-09 03:52:38.091503: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff9600 of size 256 by op Sub action_count 94195273335761 step 0 next 2479
2021-08-09 03:52:38.091527: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff9700 of size 1536 by op Fill action_count 94195455013424 step 0 next 2409
2021-08-09 03:52:38.091563: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff9d00 of size 256 by op Sub action_count 94195273335762 step 0 next 213
2021-08-09 03:52:38.091596: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbff9e00 of size 1536 by op Fill action_count 94195455013665 step 0 next 1614
2021-08-09 03:52:38.091630: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffa400 of size 512 by op Fill action_count 94195455013667 step 0 next 2799
2021-08-09 03:52:38.091652: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffa600 of size 512 by op Fill action_count 94195455013668 step 0 next 3076
2021-08-09 03:52:38.091671: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffa800 of size 512 by op Fill action_count 94195455013670 step 0 next 1341
2021-08-09 03:52:38.091690: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffaa00 of size 512 by op Fill action_count 94195455013671 step 0 next 2899
2021-08-09 03:52:38.091709: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffac00 of size 512 by op Fill action_count 94195455013672 step 0 next 2602
2021-08-09 03:52:38.091728: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffae00 of size 1024 by op Fill action_count 94195455013674 step 0 next 2624
2021-08-09 03:52:38.091747: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffb200 of size 1024 by op Fill action_count 94195455013675 step 0 next 3042
2021-08-09 03:52:38.091766: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffb600 of size 1280 by op Fill action_count 94195455013676 step 0 next 1868
2021-08-09 03:52:38.091785: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffbb00 of size 256 by op Sub action_count 94195273335803 step 0 next 536
2021-08-09 03:52:38.091804: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffbc00 of size 256 by op Sub action_count 94195273335804 step 0 next 1269
2021-08-09 03:52:38.091823: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffbd00 of size 3840 by op Fill action_count 94195455013467 step 0 next 332
2021-08-09 03:52:38.091841: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffcc00 of size 256 by op Sub action_count 94195273335775 step 0 next 403
2021-08-09 03:52:38.091858: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffcd00 of size 256 by op assert_less_equal_5/Assert/AssertGuard/pivot_f/_153 action_count 94195455013856 step 0 next 1672
2021-08-09 03:52:38.091876: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffce00 of size 256 by op assert_less_equal_5/Assert/AssertGuard/pivot_t/_154 action_count 94195455013857 step 0 next 2153
2021-08-09 03:52:38.091896: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffcf00 of size 256 by op assert_greater_equal_6/Assert/AssertGuard/pivot_f/_171 action_count 94195455013858 step 0 next 2223
2021-08-09 03:52:38.091916: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd000 of size 256 by op assert_greater_equal_6/Assert/AssertGuard/pivot_t/_172 action_count 94195455013859 step 0 next 978
2021-08-09 03:52:38.091933: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd100 of size 256 by op assert_less_equal_6/Assert/AssertGuard/pivot_f/_181 action_count 94195455013860 step 0 next 2072
2021-08-09 03:52:38.091950: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd200 of size 256 by op assert_less_equal_6/Assert/AssertGuard/pivot_t/_182 action_count 94195455013861 step 0 next 1554
2021-08-09 03:52:38.091968: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd300 of size 256 by op gradient_tape/unet_depth3/two_conv_decoder1/batch_normalization_559/moments/scalar action_count 94195455013862
step 0 next 2185
2021-08-09 03:52:38.091989: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd400 of size 256 by op assert_greater_equal_7/Assert/AssertGuard/pivot_f/_199 action_count 94195455013863 step 0 next 2208
2021-08-09 03:52:38.092011: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd500 of size 256 by op assert_greater_equal_7/Assert/AssertGuard/pivot_t/_200 action_count 94195455013864 step 0 next 1883
2021-08-09 03:52:38.092032: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd600 of size 256 by op assert_less_equal_7/Assert/AssertGuard/pivot_f/_209 action_count 94195455013865 step 0 next 2040
2021-08-09 03:52:38.092052: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd700 of size 256 by op assert_less_equal_7/Assert/AssertGuard/pivot_t/_210 action_count 94195455013866 step 0 next 2211
2021-08-09 03:52:38.092071: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd800 of size 256 by op assert_greater_equal_8/Assert/AssertGuard/pivot_f/_227 action_count 94195455013867 step 0 next 924
2021-08-09 03:52:38.092091: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffd900 of size 256 by op assert_greater_equal_8/Assert/AssertGuard/pivot_t/_228 action_count 94195455013868 step 0 next 24
2021-08-09 03:52:38.092109: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffda00 of size 256 by op gradient_tape/unet_depth3/two_conv_decoder1/batch_normalization_558/moments/scalar action_count 94195455013869
step 0 next 2162
2021-08-09 03:52:38.092131: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffdb00 of size 256 by op assert_less_equal_8/Assert/AssertGuard/pivot_f/_237 action_count 94195455013870 step 0 next 2206
2021-08-09 03:52:38.092148: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffdc00 of size 256 by op assert_less_equal_8/Assert/AssertGuard/pivot_t/_238 action_count 94195455013871 step 0 next 2085
2021-08-09 03:52:38.092166: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffdd00 of size 256 by op assert_greater_equal_9/Assert/AssertGuard/pivot_f/_255 action_count 94195455013872 step 0 next 1053
2021-08-09 03:52:38.092188: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffde00 of size 256 by op assert_greater_equal_9/Assert/AssertGuard/pivot_t/_256 action_count 94195455013873 step 0 next 2159
2021-08-09 03:52:38.092205: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffdf00 of size 256 by op assert_less_equal_9/Assert/AssertGuard/pivot_f/_265 action_count 94195455013874 step 0 next 891
2021-08-09 03:52:38.092226: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe000 of size 256 by op assert_less_equal_9/Assert/AssertGuard/pivot_t/_266 action_count 94195455013875 step 0 next 728
2021-08-09 03:52:38.092245: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe100 of size 256 by op gradient_tape/unet_depth3/conv_transpose_decoder1/batch_normalization_557/moments/scalar action_count 941954550
13876 step 0 next 2077
2021-08-09 03:52:38.092265: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe200 of size 256 by op assert_greater_equal_10/Assert/AssertGuard/pivot_f/_283 action_count 94195455013877 step 0 next 2230
2021-08-09 03:52:38.092285: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe300 of size 256 by op assert_greater_equal_10/Assert/AssertGuard/pivot_t/_284 action_count 94195455013878 step 0 next 2237
2021-08-09 03:52:38.092307: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe400 of size 256 by op assert_less_equal_10/Assert/AssertGuard/pivot_f/_293 action_count 94195455013879 step 0 next 2207
2021-08-09 03:52:38.092329: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe500 of size 256 by op assert_less_equal_10/Assert/AssertGuard/pivot_t/_294 action_count 94195455013880 step 0 next 1983
2021-08-09 03:52:38.092346: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe600 of size 256 by op assert_greater_equal_11/Assert/AssertGuard/pivot_f/_311 action_count 94195455013881 step 0 next 712
2021-08-09 03:52:38.092364: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe700 of size 256 by op assert_greater_equal_11/Assert/AssertGuard/pivot_t/_312 action_count 94195455013882 step 0 next 1848
2021-08-09 03:52:38.092381: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe800 of size 256 by op gradient_tape/unet_depth3/two_conv_decoder2/batch_normalization_556/moments/scalar action_count 94195455013883
step 0 next 2205
2021-08-09 03:52:38.092399: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffe900 of size 256 by op assert_less_equal_11/Assert/AssertGuard/pivot_f/_321 action_count 94195455013884 step 0 next 1795
2021-08-09 03:52:38.092419: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffea00 of size 256 by op assert_less_equal_11/Assert/AssertGuard/pivot_t/_322 action_count 94195455013885 step 0 next 2227
2021-08-09 03:52:38.092441: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffeb00 of size 256 by op assert_greater_equal_12/Assert/AssertGuard/pivot_f/_339 action_count 94195455013886 step 0 next 2231
2021-08-09 03:52:38.092462: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffec00 of size 256 by op assert_greater_equal_12/Assert/AssertGuard/pivot_t/_340 action_count 94195455013887 step 0 next 1767
2021-08-09 03:52:38.092483: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffed00 of size 256 by op assert_less_equal_12/Assert/AssertGuard/pivot_f/_349 action_count 94195455013888 step 0 next 1785
2021-08-09 03:52:38.092501: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffee00 of size 256 by op assert_less_equal_12/Assert/AssertGuard/pivot_t/_350 action_count 94195455013889 step 0 next 1151
2021-08-09 03:52:38.092528: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffef00 of size 256 by op gradient_tape/unet_depth3/two_conv_decoder2/batch_normalization_555/moments/scalar action_count 94195455013890
step 0 next 2139
2021-08-09 03:52:38.092562: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff000 of size 256 by op assert_greater_equal_13/Assert/AssertGuard/pivot_f/_367 action_count 94195455013891 step 0 next 192
2021-08-09 03:52:38.092595: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff100 of size 256 by op assert_greater_equal_13/Assert/AssertGuard/pivot_t/_368 action_count 94195455013892 step 0 next 2203
2021-08-09 03:52:38.092618: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff200 of size 256 by op assert_less_equal_13/Assert/AssertGuard/pivot_f/_377 action_count 94195455013893 step 0 next 2020
2021-08-09 03:52:38.092635: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff300 of size 256 by op assert_less_equal_13/Assert/AssertGuard/pivot_t/_378 action_count 94195455013894 step 0 next 2234
2021-08-09 03:52:38.092652: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff400 of size 256 by op assert_greater_equal_14/Assert/AssertGuard/pivot_f/_395 action_count 94195455013895 step 0 next 2428
2021-08-09 03:52:38.092672: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff500 of size 256 by op assert_greater_equal_14/Assert/AssertGuard/pivot_t/_396 action_count 94195455013896 step 0 next 2429
2021-08-09 03:52:38.092692: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff600 of size 256 by op assert_less_equal_14/Assert/AssertGuard/pivot_f/_405 action_count 94195455013897 step 0 next 2167
2021-08-09 03:52:38.092712: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff700 of size 256 by op assert_less_equal_14/Assert/AssertGuard/pivot_t/_406 action_count 94195455013898 step 0 next 186
2021-08-09 03:52:38.092732: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff800 of size 256 by op gradient_tape/unet_depth3/conv_transpose_decoder2/batch_normalization_554/moments/scalar action_count 941954550
13899 step 0 next 1800
2021-08-09 03:52:38.092752: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfff900 of size 256 by op assert_greater_equal_15/Assert/AssertGuard/pivot_f/_423 action_count 94195455013900 step 0 next 650
2021-08-09 03:52:38.092772: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfffa00 of size 256 by op assert_greater_equal_15/Assert/AssertGuard/pivot_t/_424 action_count 94195455013901 step 0 next 2488
2021-08-09 03:52:38.092793: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfffb00 of size 256 by op assert_less_equal_15/Assert/AssertGuard/pivot_f/_433 action_count 94195455013902 step 0 next 2124
2021-08-09 03:52:38.092815: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfffc00 of size 256 by op assert_less_equal_15/Assert/AssertGuard/pivot_t/_434 action_count 94195455013903 step 0 next 82
2021-08-09 03:52:38.092838: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfffd00 of size 256 by op assert_greater_equal_16/Assert/AssertGuard/pivot_f/_451 action_count 94195455013904 step 0 next 2047
2021-08-09 03:52:38.092856: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbfffe00 of size 256 by op assert_greater_equal_16/Assert/AssertGuard/pivot_t/_452 action_count 94195455013905 step 0 next 2183
2021-08-09 03:52:38.092876: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3abbffff00 of size 256 by op gradient_tape/unet_depth3/two_conv_center/batch_normalization_553/moments/scalar action_count 94195455013906 st
ep 0 next 18446744073709551615
2021-08-09 03:52:38.092894: I tensorflow/core/common_runtime/bfc_allocator.cc:1027] Next region of size 1073741824
2021-08-09 03:52:38.092914: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8000000 of size 3328 by op Fill action_count 94195455013519 step 0 next 2009
2021-08-09 03:52:38.092931: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8000d00 of size 256 by op Sub action_count 94195162895941 step 0 next 934
2021-08-09 03:52:38.092948: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8000e00 of size 13459968 by op Add action_count 94195455013475 step 0 next 1093
2021-08-09 03:52:38.092968: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7000 of size 512 by op Fill action_count 94195410012592 step 0 next 2129
2021-08-09 03:52:38.092987: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7200 of size 512 by op Fill action_count 94195410012593 step 0 next 2577
2021-08-09 03:52:38.093010: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7400 of size 256 by op AssignAddVariableOp_28 action_count 94195455011291 step 18076379956407770847 next 340
2021-08-09 03:52:38.093028: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7500 of size 256 by op AssignAddVariableOp_26 action_count 94195455011239 step 18076379956407770847 next 1402
2021-08-09 03:52:38.093051: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7600 of size 256 by op AssignAddVariableOp_36 action_count 94195455011295 step 18076379956407770847 next 2803
2021-08-09 03:52:38.093069: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7700 of size 256 by op AssignAddVariableOp_34 action_count 94195455011251 step 18076379956407770847 next 183
2021-08-09 03:52:38.093089: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7800 of size 256 by op Fill action_count 94195410012556 step 0 next 758
2021-08-09 03:52:38.093110: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7900 of size 256 by op AssignVariableOp action_count 94195454831374 step 0 next 810
2021-08-09 03:52:38.093130: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7a00 of size 256 by op Fill action_count 94195410012572 step 0 next 253
2021-08-09 03:52:38.093149: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7b00 of size 256 by op Fill action_count 94195410012880 step 0 next 2035
2021-08-09 03:52:38.093168: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7c00 of size 256 by op AssignVariableOp action_count 94195454831376 step 0 next 621
2021-08-09 03:52:38.093188: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7d00 of size 512 by op Fill action_count 94195410012594 step 0 next 809
2021-08-09 03:52:38.093207: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd7f00 of size 512 by op Fill action_count 94195410012595 step 0 next 840
2021-08-09 03:52:38.093226: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd8100 of size 512 by op Fill action_count 94195410012596 step 0 next 1291
2021-08-09 03:52:38.093242: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd8300 of size 256 by op Sub action_count 94195455013457 step 0 next 1971
2021-08-09 03:52:38.093258: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd8400 of size 768 by op Fill action_count 94195410012607 step 0 next 1035
2021-08-09 03:52:38.093274: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd8700 of size 768 by op Fill action_count 94195410012608 step 0 next 2087
2021-08-09 03:52:38.093290: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd8a00 of size 1024 by op Fill action_count 94195410012616 step 0 next 1381
2021-08-09 03:52:38.093306: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd8e00 of size 2048 by op Fill action_count 94195467563027 step 0 next 1329
2021-08-09 03:52:38.093322: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd9600 of size 1792 by op Fill action_count 94195467563035 step 0 next 266
2021-08-09 03:52:38.093342: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cd9d00 of size 1792 by op Fill action_count 94195467563036 step 0 next 2370
2021-08-09 03:52:38.093362: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cda400 of size 1792 by op Fill action_count 94195467563037 step 0 next 2210
2021-08-09 03:52:38.093382: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdab00 of size 1792 by op Fill action_count 94195467563038 step 0 next 3059
2021-08-09 03:52:38.093401: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdb200 of size 1792 by op Fill action_count 94195467563039 step 0 next 595
2021-08-09 03:52:38.093420: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdb900 of size 1792 by op Fill action_count 94195467563047 step 0 next 2412
2021-08-09 03:52:38.093439: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdc000 of size 1792 by op Fill action_count 94195467563048 step 0 next 2597
2021-08-09 03:52:38.093459: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdc700 of size 1792 by op Fill action_count 94195467563049 step 0 next 2039
2021-08-09 03:52:38.093483: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdce00 of size 1792 by op Fill action_count 94195467563050 step 0 next 2016
2021-08-09 03:52:38.093503: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdd500 of size 1792 by op Fill action_count 94195467563051 step 0 next 2422
2021-08-09 03:52:38.093526: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cddc00 of size 1792 by op Fill action_count 94195467563059 step 0 next 789
2021-08-09 03:52:38.093546: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cde300 of size 1792 by op Fill action_count 94195467563060 step 0 next 1337
2021-08-09 03:52:38.093567: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdea00 of size 1792 by op Fill action_count 94195467563061 step 0 next 2351
2021-08-09 03:52:38.093583: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdf100 of size 1792 by op Fill action_count 94195467563062 step 0 next 1763
2021-08-09 03:52:38.093603: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdf800 of size 1792 by op Fill action_count 94195467563063 step 0 next 732
2021-08-09 03:52:38.093619: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8cdff00 of size 1024 by op Fill action_count 94195467563071 step 0 next 2430
2021-08-09 03:52:38.093635: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce0300 of size 1024 by op Fill action_count 94195467563072 step 0 next 1969
2021-08-09 03:52:38.093651: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce0700 of size 1024 by op Fill action_count 94195467563073 step 0 next 1127
2021-08-09 03:52:38.093671: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce0b00 of size 1024 by op Fill action_count 94195467563074 step 0 next 1752
2021-08-09 03:52:38.093690: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce0f00 of size 1024 by op Fill action_count 94195467563075 step 0 next 1139
2021-08-09 03:52:38.093707: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce1300 of size 1024 by op Fill action_count 94195467563083 step 0 next 1652
2021-08-09 03:52:38.093726: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce1700 of size 768 by op Fill action_count 94195410012582 step 0 next 1429
2021-08-09 03:52:38.093748: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce1a00 of size 256 by op AssignAddVariableOp_11 action_count 94195455011217 step 18076379956407770847 next 2936
2021-08-09 03:52:38.093769: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce1b00 of size 256 by op AssignAddVariableOp_20 action_count 94195455011287 step 18076379956407770847 next 1330
2021-08-09 03:52:38.093787: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce1c00 of size 256 by op AssignAddVariableOp_18 action_count 94195455011227 step 18076379956407770847 next 2503
2021-08-09 03:52:38.093807: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce1d00 of size 256 by op Fill action_count 94195410012557 step 0 next 1716
2021-08-09 03:52:38.093827: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce1e00 of size 256 by op AssignVariableOp action_count 94195454831386 step 0 next 1521
2021-08-09 03:52:38.093843: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce1f00 of size 256 by op Fill action_count 94195410012558 step 0 next 2619
2021-08-09 03:52:38.093863: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce2000 of size 512 by op Fill action_count 94195455013397 step 0 next 1784
2021-08-09 03:52:38.093882: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce2200 of size 512 by op Fill action_count 94195455013398 step 0 next 1957
2021-08-09 03:52:38.093898: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce2400 of size 512 by op Fill action_count 94195410012580 step 0 next 1687
2021-08-09 03:52:38.093914: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce2600 of size 1536 by op Fill action_count 94195455013425 step 0 next 864
2021-08-09 03:52:38.093930: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce2c00 of size 768 by op Fill action_count 94195410012617 step 0 next 1205
2021-08-09 03:52:38.093946: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce2f00 of size 768 by op Fill action_count 94195410012618 step 0 next 2439
2021-08-09 03:52:38.093962: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce3200 of size 768 by op Fill action_count 94195410012619 step 0 next 328
2021-08-09 03:52:38.093982: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce3500 of size 768 by op Fill action_count 94195410012620 step 0 next 1959
2021-08-09 03:52:38.093998: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce3800 of size 2304 by op Fill action_count 94195410012628 step 0 next 806
2021-08-09 03:52:38.094017: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce4100 of size 256 by op Sub action_count 94195455013470 step 0 next 807
2021-08-09 03:52:38.094037: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac8ce4200 of size 6291456 by op Add action_count 94195410012673 step 0 next 285
2021-08-09 03:52:38.094056: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac92e4200 of size 6291456 by op Fill action_count 94195410012975 step 0 next 2275
2021-08-09 03:52:38.094073: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ac98e4200 of size 18874368 by op Fill action_count 94195410012979 step 0 next 766
2021-08-09 03:52:38.094092: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acaae4200 of size 12582912 by op Add action_count 94195410012685 step 0 next 2480
2021-08-09 03:52:38.094109: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acb6e4200 of size 3145728 by op Add action_count 94195410012757 step 0 next 1350
2021-08-09 03:52:38.094128: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acb9e4200 of size 3145728 by op Fill action_count 94195410012971 step 0 next 1449
2021-08-09 03:52:38.094148: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acbce4200 of size 6291456 by op Add action_count 94195410012709 step 0 next 1797
2021-08-09 03:52:38.094164: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acc2e4200 of size 4767744 by op Fill action_count 94195410012983 step 0 next 1516
2021-08-09 03:52:38.094184: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acc770200 of size 7193600 by op Add action_count 94195410012745 step 0 next 1899
2021-08-09 03:52:38.094200: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acce4c600 of size 3072 by op Fill action_count 94195455013509 step 0 next 140
2021-08-09 03:52:38.094217: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acce4d200 of size 256 by op Sub action_count 94195347535784 step 0 next 80
2021-08-09 03:52:38.094236: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acce4d300 of size 256 by op Sub action_count 94195347535785 step 0 next 2606
2021-08-09 03:52:38.094252: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acce4d400 of size 256 by op Sub action_count 94195347535798 step 0 next 174
2021-08-09 03:52:38.094272: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acce4d500 of size 256 by op Sub action_count 94195347535799 step 0 next 1629
2021-08-09 03:52:38.094291: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acce4d600 of size 256 by op Sub action_count 94195347535824 step 0 next 2022
2021-08-09 03:52:38.094310: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acce4d700 of size 2198272 by op Add action_count 94195455013555 step 0 next 3068
2021-08-09 03:52:38.094327: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd066200 of size 824576 by op Add action_count 94195455013593 step 0 next 2543
2021-08-09 03:52:38.094346: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd12f700 of size 1532928 by op Add action_count 94195455013581 step 0 next 237
2021-08-09 03:52:38.094366: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd2a5b00 of size 256 by op Sub action_count 94195347535825 step 0 next 342
2021-08-09 03:52:38.094382: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd2a5c00 of size 2048 by op Fill action_count 94195455013522 step 0 next 2807
2021-08-09 03:52:38.094401: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd2a6400 of size 256 by op Sub action_count 94195455013524 step 0 next 2157
2021-08-09 03:52:38.094420: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd2a6500 of size 256 by op Sub action_count 94195455013525 step 0 next 2992
2021-08-09 03:52:38.094440: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd2a6600 of size 2304 by op Fill action_count 94195455013532 step 0 next 1806
2021-08-09 03:52:38.094459: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd2a6f00 of size 256 by op Sub action_count 94195365012495 step 0 next 2064
2021-08-09 03:52:38.094480: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd2a7000 of size 4046080 by op Add action_count 94195455013447 step 0 next 364
2021-08-09 03:52:38.094497: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd682d00 of size 256 by op Sub action_count 94195347535729 step 0 next 385
2021-08-09 03:52:38.094512: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd682e00 of size 256 by op Sub action_count 94195347535742 step 0 next 2240
2021-08-09 03:52:38.094542: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd682f00 of size 1024 by op Fill action_count 94195455013572 step 0 next 3089
2021-08-09 03:52:38.094563: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd683300 of size 1024 by op Fill action_count 94195455013573 step 0 next 613
2021-08-09 03:52:38.094582: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd683700 of size 1024 by op Fill action_count 94195455013574 step 0 next 1338
2021-08-09 03:52:38.094602: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd683b00 of size 256 by op Sub action_count 94195455013576 step 0 next 1873
2021-08-09 03:52:38.094618: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd683c00 of size 256 by op Sub action_count 94195455013577 step 0 next 792
2021-08-09 03:52:38.094638: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd683d00 of size 1024 by op Fill action_count 94195455013584 step 0 next 2937
2021-08-09 03:52:38.094657: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd684100 of size 1280 by op Fill action_count 94195455013585 step 0 next 1544
2021-08-09 03:52:38.094677: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd684600 of size 256 by op Sub action_count 94195347535743 step 0 next 604
2021-08-09 03:52:38.094697: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd684700 of size 256 by op Sub action_count 94195347535756 step 0 next 1271
2021-08-09 03:52:38.094716: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acd684800 of size 14024704 by op Add action_count 94195455013489 step 0 next 2985
2021-08-09 03:52:38.094733: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ace3e4800 of size 5775360 by op Fill action_count 94195455013701 step 0 next 1295
2021-08-09 03:52:38.094752: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ace966800 of size 7012352 by op Fill action_count 94195455013709 step 0 next 1346
2021-08-09 03:52:38.094771: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acf016800 of size 3297536 by op Fill action_count 94195455013713 step 0 next 2856
2021-08-09 03:52:38.094788: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acf33b900 of size 2198272 by op Fill action_count 94195455013717 step 0 next 1339
2021-08-09 03:52:38.094806: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acf554400 of size 2930944 by op Fill action_count 94195455013721 step 0 next 3051
2021-08-09 03:52:38.094825: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acf81fd00 of size 1099264 by op Fill action_count 94195455013751 step 0 next 1099
2021-08-09 03:52:38.094844: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acf92c300 of size 2198272 by op Fill action_count 94195455013755 step 0 next 2264
2021-08-09 03:52:38.094864: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acfb44e00 of size 4396544 by op Fill action_count 94195455013759 step 0 next 1717
2021-08-09 03:52:38.094881: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3acff76400 of size 8792832 by op Fill action_count 94195455013763 step 0 next 1770
2021-08-09 03:52:38.094901: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad07d8f00 of size 10772736 by op Fill action_count 94195455013771 step 0 next 1790
2021-08-09 03:52:38.094921: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad121f000 of size 256 by op Sub action_count 94195273335776 step 0 next 825
2021-08-09 03:52:38.094941: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad121f100 of size 256 by op Sub action_count 94195455013538 step 0 next 566
2021-08-09 03:52:38.094960: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad121f200 of size 256 by op Sub action_count 94195455013539 step 0 next 3050
2021-08-09 03:52:38.094979: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad121f300 of size 1792 by op Fill action_count 94195455013546 step 0 next 197
2021-08-09 03:52:38.094995: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad121fa00 of size 3072 by op Fill action_count 94195455013547 step 0 next 1496
2021-08-09 03:52:38.095015: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad1220600 of size 256 by op Sub action_count 94195273335789 step 0 next 908
2021-08-09 03:52:38.095035: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad1220700 of size 256 by op Sub action_count 94195273335790 step 0 next 744
2021-08-09 03:52:38.095055: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad1220800 of size 256 by op Adam/Pow action_count 94195480111529 step 13755587135625806913 next 3045
2021-08-09 03:52:38.095094: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad1220900 of size 256 by op Adam/Cast action_count 94195480111527 step 13755587135625806913 next 2731
2021-08-09 03:52:38.095116: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad1220a00 of size 1441792 by op SameWorkerRecvDone action_count 94195480111530 step 0 next 3056
2021-08-09 03:52:38.095136: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad1380a00 of size 1441792 by op SameWorkerRecvDone action_count 94195480111531 step 0 next 2916
2021-08-09 03:52:38.095153: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad14e0a00 of size 1441792 by op binary_ce_dice/mul action_count 94195480111532 step 13755587135625806913 next 2802
2021-08-09 03:52:38.095174: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad1640a00 of size 360448 by op Cast_4 action_count 94195480111533 step 13755587135625806913 next 2990
2021-08-09 03:52:38.095191: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad1698a00 of size 360448 by op LogicalNot action_count 94195480111534 step 13755587135625806913 next 2876
2021-08-09 03:52:38.095209: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad16f0a00 of size 36044800 by op Tile_92 action_count 94195480111535 step 13755587135625806913 next 3077
2021-08-09 03:52:38.095230: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad3950a00 of size 36044800 by op LogicalNot_31 action_count 94195480111536 step 13755587135625806913 next 2974
2021-08-09 03:52:38.095247: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad5bb0a00 of size 1441792 by op gradient_tape/binary_ce_dice/logistic_loss/mul/Mul action_count 94195480111537 step 13755587135625806913 ne
xt 2846
2021-08-09 03:52:38.095267: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad5d10a00 of size 512 by op unet_depth4/encode0/batch_normalization_580/moments/mean action_count 94195480111555 step 13755587135625806913
next 3067
2021-08-09 03:52:38.095288: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad5d10c00 of size 512 by op unet_depth4/encode0/batch_normalization_580/batchnorm/mul action_count 94195480111567 step 13755587135625806913
 next 3027
2021-08-09 03:52:38.095305: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ad5d10e00 of size 245764096 by op unet_depth4/encode0/batch_normalization_580/batchnorm/mul_1 action_count 94195480111568 step 137555871356
25806913 next 1011
2021-08-09 03:52:38.095322: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4771e00 of size 256 by op Sub action_count 94195129464801 step 0 next 1921
2021-08-09 03:52:38.095340: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4771f00 of size 256 by op Sub action_count 94195129464802 step 0 next 2051
2021-08-09 03:52:38.095360: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4772000 of size 1024 by op Fill action_count 94195455013678 step 0 next 2527
2021-08-09 03:52:38.095379: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4772400 of size 1024 by op Fill action_count 94195455013679 step 0 next 2929
2021-08-09 03:52:38.095398: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4772800 of size 1024 by op Fill action_count 94195455013680 step 0 next 2596
2021-08-09 03:52:38.095414: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4772c00 of size 1792 by op Fill action_count 94195455013682 step 0 next 1426
2021-08-09 03:52:38.095434: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4773300 of size 2560 by op Fill action_count 94195455013683 step 0 next 2441
2021-08-09 03:52:38.095453: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4773d00 of size 256 by op Sub action_count 94195299052817 step 0 next 906
2021-08-09 03:52:38.095473: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4773e00 of size 256 by op AssignVariableOp action_count 94195454831358 step 0 next 1573
2021-08-09 03:52:38.095492: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4773f00 of size 256 by op Sub action_count 94195299052818 step 0 next 2171
2021-08-09 03:52:38.095508: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4774000 of size 2816 by op Fill action_count 94195455013507 step 0 next 2490
2021-08-09 03:52:38.095531: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4774b00 of size 256 by op Sub action_count 94195299052831 step 0 next 399
2021-08-09 03:52:38.095550: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4774c00 of size 1280 by op Fill action_count 94195455013411 step 0 next 748
2021-08-09 03:52:38.095569: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4775100 of size 256 by op Sub action_count 94195299052832 step 0 next 1298
2021-08-09 03:52:38.095588: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4775200 of size 2304 by op Fill action_count 94195455013493 step 0 next 1658
2021-08-09 03:52:38.095604: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4775b00 of size 256 by op Sub action_count 94195365012481 step 0 next 505
2021-08-09 03:52:38.095624: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4775c00 of size 2816 by op Fill action_count 94195455013508 step 0 next 1954
2021-08-09 03:52:38.095643: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4776700 of size 256 by op Sub action_count 94195129464851 step 0 next 2154
2021-08-09 03:52:38.095662: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4776800 of size 256 by op Sub action_count 94195129464852 step 0 next 1941
2021-08-09 03:52:38.095681: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4776900 of size 3584 by op Fill action_count 94195455013690 step 0 next 2822
2021-08-09 03:52:38.095697: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4777700 of size 4096 by op Fill action_count 94195455013691 step 0 next 2004
2021-08-09 03:52:38.095717: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4778700 of size 256 by op Sub action_count 94195129464865 step 0 next 2138
2021-08-09 03:52:38.095735: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4778800 of size 256 by op Sub action_count 94195129464866 step 0 next 1897
2021-08-09 03:52:38.095755: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4778900 of size 205568 by op Fill action_count 94195467563236 step 0 next 1042
2021-08-09 03:52:38.095771: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47aac00 of size 256 by op Sub action_count 94195129464999 step 0 next 2123
2021-08-09 03:52:38.095791: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47aad00 of size 256 by op Sub action_count 94195129465000 step 0 next 1919
2021-08-09 03:52:38.095810: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47aae00 of size 768 by op Fill action_count 94195480110828 step 0 next 2305
2021-08-09 03:52:38.095830: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ab100 of size 256 by op AssignAddVariableOp_27 action_count 94195480109387 step 17368542405085625385 next 2028
2021-08-09 03:52:38.095852: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ab200 of size 256 by op AssignAddVariableOp_29 action_count 94195480109439 step 17368542405085625385 next 740
2021-08-09 03:52:38.095873: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ab300 of size 256 by op AssignAddVariableOp_35 action_count 94195480109399 step 17368542405085625385 next 324
2021-08-09 03:52:38.095890: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ab400 of size 512 by op Fill action_count 94195480110826 step 0 next 2241
2021-08-09 03:52:38.095911: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ab600 of size 256 by op AssignAddVariableOp_37 action_count 94195480109443 step 17368542405085625385 next 892
2021-08-09 03:52:38.095928: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ab700 of size 256 by op AssignAddVariableOp_4 action_count 94195480109414 step 17368542405085625385 next 413
2021-08-09 03:52:38.095950: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ab800 of size 256 by op AssignAddVariableOp_2 action_count 94195480109331 step 17368542405085625385 next 1324
2021-08-09 03:52:38.095966: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ab900 of size 512 by op Fill action_count 94195480110827 step 0 next 957
2021-08-09 03:52:38.095990: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47abb00 of size 256 by op AssignAddVariableOp_5 action_count 94195480109416 step 17368542405085625385 next 1237
2021-08-09 03:52:38.096009: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47abc00 of size 256 by op Sub action_count 94195480110871 step 0 next 917
2021-08-09 03:52:38.096029: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47abd00 of size 256 by op AssignAddVariableOp_26 action_count 94195480109385 step 17368542405085625385 next 2161
2021-08-09 03:52:38.096045: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47abe00 of size 256 by op AssignAddVariableOp_36 action_count 94195480109441 step 17368542405085625385 next 1557
2021-08-09 03:52:38.096066: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47abf00 of size 256 by op AssignAddVariableOp_34 action_count 94195480109397 step 17368542405085625385 next 2251
2021-08-09 03:52:38.096083: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ac000 of size 1024 by op Fill action_count 94195480110865 step 0 next 181
2021-08-09 03:52:38.096099: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ac400 of size 1024 by op Fill action_count 94195480110866 step 0 next 1358
2021-08-09 03:52:38.096119: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ac800 of size 1024 by op Fill action_count 94195480110867 step 0 next 1461
2021-08-09 03:52:38.096138: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47acc00 of size 1024 by op Fill action_count 94195480110868 step 0 next 2290
2021-08-09 03:52:38.096154: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ad000 of size 2048 by op Fill action_count 94195480110878 step 0 next 68
2021-08-09 03:52:38.096170: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ad800 of size 2048 by op Fill action_count 94195480110879 step 0 next 314
2021-08-09 03:52:38.096189: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ae000 of size 2048 by op Fill action_count 94195480110880 step 0 next 1031
2021-08-09 03:52:38.096207: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ae800 of size 3072 by op Fill action_count 94195480110881 step 0 next 2388
2021-08-09 03:52:38.096227: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47af400 of size 256 by op AssignVariableOp action_count 94195480053214 step 0 next 256
2021-08-09 03:52:38.096243: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47af500 of size 512 by op Fill action_count 94195480110838 step 0 next 1188
2021-08-09 03:52:38.096262: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47af700 of size 512 by op Fill action_count 94195480110839 step 0 next 123
2021-08-09 03:52:38.096282: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47af900 of size 512 by op Fill action_count 94195480110840 step 0 next 1211
2021-08-09 03:52:38.096301: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47afb00 of size 1536 by op Add action_count 94195480110821 step 0 next 848
2021-08-09 03:52:38.096320: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b0100 of size 1024 by op Fill action_count 94195480110852 step 0 next 414
2021-08-09 03:52:38.096339: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b0500 of size 1024 by op Fill action_count 94195480110853 step 0 next 2495
2021-08-09 03:52:38.096355: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b0900 of size 1024 by op Fill action_count 94195480110854 step 0 next 1517
2021-08-09 03:52:38.096374: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b0d00 of size 1024 by op Fill action_count 94195480110864 step 0 next 1878
2021-08-09 03:52:38.096393: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b1100 of size 256 by op AssignVariableOp action_count 94195480053222 step 0 next 743
2021-08-09 03:52:38.096409: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b1200 of size 256 by op AssignVariableOp action_count 94195480053224 step 0 next 637
2021-08-09 03:52:38.096428: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b1300 of size 256 by op AssignVariableOp action_count 94195480053228 step 0 next 261
2021-08-09 03:52:38.096445: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b1400 of size 2048 by op Fill action_count 94195480110882 step 0 next 667
2021-08-09 03:52:38.096464: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b1c00 of size 2048 by op Fill action_count 94195480110890 step 0 next 796
2021-08-09 03:52:38.096483: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b2400 of size 2048 by op Fill action_count 94195480110891 step 0 next 262
2021-08-09 03:52:38.096499: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b2c00 of size 2048 by op Fill action_count 94195480110892 step 0 next 383
2021-08-09 03:52:38.096525: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b3400 of size 2048 by op Fill action_count 94195480110893 step 0 next 1383
2021-08-09 03:52:38.096560: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b3c00 of size 2048 by op Fill action_count 94195480110894 step 0 next 249
2021-08-09 03:52:38.096592: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b4400 of size 2048 by op Fill action_count 94195480110904 step 0 next 26
2021-08-09 03:52:38.096612: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b4c00 of size 2048 by op Fill action_count 94195480110905 step 0 next 421
2021-08-09 03:52:38.096632: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b5400 of size 2048 by op Fill action_count 94195480110906 step 0 next 1905
2021-08-09 03:52:38.096648: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b5c00 of size 2816 by op Fill action_count 94195480110907 step 0 next 568
2021-08-09 03:52:38.096671: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b6700 of size 256 by op binary_ce_dice/weighted_loss/num_elements/Cast action_count 94195468133245 step 0 next 896
2021-08-09 03:52:38.096690: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b6800 of size 512 by op Reshape_92 action_count 94195468133247 step 0 next 1389
2021-08-09 03:52:38.096709: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b6a00 of size 256 by op Cast_181/x action_count 94195468133253 step 0 next 2512
2021-08-09 03:52:38.096726: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b6b00 of size 256 by op unet_depth3/encode0/batch_normalization_563/batchnorm/add/y action_count 94195468133254 step 0 next 1801
2021-08-09 03:52:38.096748: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b6c00 of size 256 by op Cast_180/x action_count 94195468133255 step 0 next 1841
2021-08-09 03:52:38.096770: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b6d00 of size 256 by op assert_greater_equal/Assert/AssertGuard/pivot_f/_3 action_count 94195468133256 step 0 next 1181
2021-08-09 03:52:38.096791: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b6e00 of size 256 by op assert_greater_equal/Assert/AssertGuard/pivot_t/_4 action_count 94195468133257 step 0 next 368
2021-08-09 03:52:38.096813: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b6f00 of size 256 by op Const_35 action_count 94195468133258 step 0 next 1382
2021-08-09 03:52:38.096834: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7000 of size 256 by op assert_less_equal/Assert/AssertGuard/pivot_f/_13 action_count 94195468133259 step 0 next 1314
2021-08-09 03:52:38.096851: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7100 of size 256 by op assert_less_equal/Assert/AssertGuard/pivot_t/_14 action_count 94195468133260 step 0 next 1702
2021-08-09 03:52:38.096872: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7200 of size 256 by op assert_greater_equal_1/Assert/AssertGuard/pivot_f/_31 action_count 94195468133261 step 0 next 180
2021-08-09 03:52:38.096892: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7300 of size 256 by op assert_greater_equal_1/Assert/AssertGuard/pivot_t/_32 action_count 94195468133262 step 0 next 2133
2021-08-09 03:52:38.096912: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7400 of size 256 by op assert_less_equal_1/Assert/AssertGuard/pivot_f/_41 action_count 94195468133263 step 0 next 15
2021-08-09 03:52:38.096931: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7500 of size 256 by op assert_less_equal_1/Assert/AssertGuard/pivot_t/_42 action_count 94195468133264 step 0 next 434
2021-08-09 03:52:38.096948: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7600 of size 256 by op assert_greater_equal_2/Assert/AssertGuard/pivot_f/_59 action_count 94195468133265 step 0 next 1844
2021-08-09 03:52:38.096969: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7700 of size 256 by op assert_greater_equal_2/Assert/AssertGuard/pivot_t/_60 action_count 94195468133266 step 0 next 310
2021-08-09 03:52:38.096987: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7800 of size 256 by op assert_less_equal_2/Assert/AssertGuard/pivot_f/_69 action_count 94195468133267 step 0 next 352
2021-08-09 03:52:38.097008: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7900 of size 256 by op assert_less_equal_2/Assert/AssertGuard/pivot_t/_70 action_count 94195468133268 step 0 next 1970
2021-08-09 03:52:38.097029: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7a00 of size 256 by op assert_greater_equal_3/Assert/AssertGuard/pivot_f/_87 action_count 94195468133269 step 0 next 894
2021-08-09 03:52:38.097046: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7b00 of size 256 by op assert_greater_equal_3/Assert/AssertGuard/pivot_t/_88 action_count 94195468133270 step 0 next 30
2021-08-09 03:52:38.097066: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7c00 of size 256 by op assert_less_equal_3/Assert/AssertGuard/pivot_f/_97 action_count 94195468133271 step 0 next 1882
2021-08-09 03:52:38.097083: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7d00 of size 256 by op assert_less_equal_3/Assert/AssertGuard/pivot_t/_98 action_count 94195468133272 step 0 next 300
2021-08-09 03:52:38.097101: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7e00 of size 256 by op assert_greater_equal_4/Assert/AssertGuard/pivot_f/_115 action_count 94195468133273 step 0 next 936
2021-08-09 03:52:38.097123: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b7f00 of size 256 by op assert_greater_equal_4/Assert/AssertGuard/pivot_t/_116 action_count 94195468133274 step 0 next 2197
2021-08-09 03:52:38.097144: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b8000 of size 256 by op assert_less_equal_4/Assert/AssertGuard/pivot_f/_125 action_count 94195468133275 step 0 next 1837
2021-08-09 03:52:38.097162: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b8100 of size 256 by op assert_less_equal_4/Assert/AssertGuard/pivot_t/_126 action_count 94195468133276 step 0 next 1143
2021-08-09 03:52:38.097182: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b8200 of size 256 by op AssignAddVariableOp_52 action_count 94195467564196 step 10280291020125615045 next 998
2021-08-09 03:52:38.097202: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b8300 of size 256 by op Sub action_count 94195480110896 step 0 next 645
2021-08-09 03:52:38.097220: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b8400 of size 256 by op Sub action_count 94195339127952 step 0 next 1973
2021-08-09 03:52:38.097240: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b8500 of size 3584 by op Fill action_count 94195455013464 step 0 next 1432
2021-08-09 03:52:38.097259: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9300 of size 256 by op Sub action_count 94195339127981 step 0 next 178
2021-08-09 03:52:38.097275: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9400 of size 256 by op Sub action_count 94195339127994 step 0 next 925
2021-08-09 03:52:38.097294: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9500 of size 256 by op Sub action_count 94195455013471 step 0 next 620
2021-08-09 03:52:38.097313: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9600 of size 256 by op AssignVariableOp action_count 94195454831394 step 0 next 1634
2021-08-09 03:52:38.097329: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9700 of size 256 by op Sub action_count 94195339127953 step 0 next 1683
2021-08-09 03:52:38.097345: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9800 of size 256 by op Sub action_count 94195455013484 step 0 next 2419
2021-08-09 03:52:38.097361: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9900 of size 256 by op Sub action_count 94195339128020 step 0 next 2170
2021-08-09 03:52:38.097376: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9a00 of size 256 by op Sub action_count 94195339128021 step 0 next 210
2021-08-09 03:52:38.097396: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9b00 of size 256 by op Sub action_count 94195365012496 step 0 next 1286
2021-08-09 03:52:38.097416: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47b9c00 of size 1536 by op Fill action_count 94195455013426 step 0 next 45
2021-08-09 03:52:38.097432: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ba200 of size 256 by op Sub action_count 94195365012569 step 0 next 2105
2021-08-09 03:52:38.097448: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ba300 of size 256 by op Sub action_count 94195365012570 step 0 next 1869
2021-08-09 03:52:38.097467: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ba400 of size 2048 by op Fill action_count 94195455013707 step 0 next 3084
2021-08-09 03:52:38.097486: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bac00 of size 2048 by op Fill action_count 94195455013708 step 0 next 2958
2021-08-09 03:52:38.097502: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bb400 of size 1792 by op Fill action_count 94195455013710 step 0 next 2896
2021-08-09 03:52:38.097526: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bbb00 of size 2304 by op Fill action_count 94195455013711 step 0 next 547
2021-08-09 03:52:38.097542: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bc400 of size 256 by op Sub action_count 94195365012482 step 0 next 2659
2021-08-09 03:52:38.097561: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bc500 of size 256 by op Sub action_count 94195455013485 step 0 next 2155
2021-08-09 03:52:38.097580: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bc600 of size 256 by op Sub action_count 94195129465085 step 0 next 1069
2021-08-09 03:52:38.097599: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bc700 of size 256 by op Sub action_count 94195129465086 step 0 next 2037
2021-08-09 03:52:38.097618: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bc800 of size 256 by op AssignVariableOp action_count 94195467505299 step 0 next 2374
2021-08-09 03:52:38.097638: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bc900 of size 256 by op AssignVariableOp action_count 94195467505301 step 0 next 1012
2021-08-09 03:52:38.097654: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bca00 of size 256 by op AssignVariableOp action_count 94195467505303 step 0 next 847
2021-08-09 03:52:38.097673: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bcb00 of size 256 by op Sub action_count 94195480110897 step 0 next 1660
2021-08-09 03:52:38.097693: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bcc00 of size 256 by op AssignVariableOp action_count 94195467505329 step 0 next 1388
2021-08-09 03:52:38.097712: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bcd00 of size 256 by op AssignVariableOp action_count 94195467505331 step 0 next 2156
2021-08-09 03:52:38.097728: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bce00 of size 1024 by op Fill action_count 94195467562927 step 0 next 54
2021-08-09 03:52:38.097744: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bd200 of size 256 by op Sum_1 action_count 94195455584270 step 0 next 1890
2021-08-09 03:52:38.097760: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bd300 of size 256 by op truediv/y action_count 94195455584272 step 0 next 511
2021-08-09 03:52:38.097782: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bd400 of size 256 by op binary_ce_dice/add_1/y action_count 94195455584273 step 0 next 1136
2021-08-09 03:52:38.097800: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bd500 of size 256 by op binary_ce_dice/weighted_loss/num_elements/Cast action_count 94195455584274 step 0 next 1994
2021-08-09 03:52:38.097820: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bd600 of size 1792 by op Fill action_count 94195467562951 step 0 next 1932
2021-08-09 03:52:38.097840: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bdd00 of size 1792 by op Fill action_count 94195467562952 step 0 next 2316
2021-08-09 03:52:38.097859: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47be400 of size 1792 by op Fill action_count 94195467562953 step 0 next 2172
2021-08-09 03:52:38.097875: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47beb00 of size 1792 by op Fill action_count 94195467562954 step 0 next 2921
2021-08-09 03:52:38.097893: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bf200 of size 1792 by op Fill action_count 94195467562955 step 0 next 2489
2021-08-09 03:52:38.097913: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47bf900 of size 2048 by op Fill action_count 94195467562963 step 0 next 2348
2021-08-09 03:52:38.097933: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0100 of size 256 by op assert_less_equal_2/Assert/AssertGuard/pivot_f/_69 action_count 94195455584296 step 0 next 1946
2021-08-09 03:52:38.097950: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0200 of size 256 by op assert_less_equal_2/Assert/AssertGuard/pivot_t/_70 action_count 94195455584297 step 0 next 1886
2021-08-09 03:52:38.097969: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0300 of size 256 by op assert_greater_equal_3/Assert/AssertGuard/pivot_f/_87 action_count 94195455584298 step 0 next 1458
2021-08-09 03:52:38.097988: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0400 of size 256 by op assert_greater_equal_3/Assert/AssertGuard/pivot_t/_88 action_count 94195455584299 step 0 next 2400
2021-08-09 03:52:38.098007: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0500 of size 256 by op assert_less_equal_3/Assert/AssertGuard/pivot_f/_97 action_count 94195455584300 step 0 next 1070
2021-08-09 03:52:38.098024: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0600 of size 256 by op assert_less_equal_3/Assert/AssertGuard/pivot_t/_98 action_count 94195455584301 step 0 next 2291
2021-08-09 03:52:38.098043: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0700 of size 256 by op assert_greater_equal_4/Assert/AssertGuard/pivot_f/_115 action_count 94195455584302 step 0 next 1394
2021-08-09 03:52:38.098060: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0800 of size 256 by op assert_greater_equal_4/Assert/AssertGuard/pivot_t/_116 action_count 94195455584303 step 0 next 2324
2021-08-09 03:52:38.098079: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0900 of size 256 by op assert_less_equal_4/Assert/AssertGuard/pivot_f/_125 action_count 94195455584304 step 0 next 2372
2021-08-09 03:52:38.098098: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0a00 of size 256 by op assert_less_equal_4/Assert/AssertGuard/pivot_t/_126 action_count 94195455584305 step 0 next 2271
2021-08-09 03:52:38.098114: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0b00 of size 256 by op assert_greater_equal_5/Assert/AssertGuard/pivot_f/_143 action_count 94195455584306 step 0 next 1843
2021-08-09 03:52:38.098134: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0c00 of size 256 by op assert_greater_equal_5/Assert/AssertGuard/pivot_t/_144 action_count 94195455584307 step 0 next 2416
2021-08-09 03:52:38.098155: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0d00 of size 256 by op assert_less_equal_5/Assert/AssertGuard/pivot_f/_153 action_count 94195455584308 step 0 next 2407
2021-08-09 03:52:38.098175: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0e00 of size 256 by op assert_less_equal_5/Assert/AssertGuard/pivot_t/_154 action_count 94195455584309 step 0 next 1783
2021-08-09 03:52:38.098197: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c0f00 of size 256 by op assert_greater_equal_6/Assert/AssertGuard/pivot_f/_171 action_count 94195455584310 step 0 next 2274
2021-08-09 03:52:38.098218: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1000 of size 256 by op assert_greater_equal_6/Assert/AssertGuard/pivot_t/_172 action_count 94195455584311 step 0 next 2379
2021-08-09 03:52:38.098236: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1100 of size 256 by op assert_less_equal_6/Assert/AssertGuard/pivot_f/_181 action_count 94195455584312 step 0 next 2435
2021-08-09 03:52:38.098255: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1200 of size 256 by op assert_less_equal_6/Assert/AssertGuard/pivot_t/_182 action_count 94195455584313 step 0 next 2280
2021-08-09 03:52:38.098272: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1300 of size 256 by op assert_greater_equal_7/Assert/AssertGuard/pivot_f/_199 action_count 94195455584314 step 0 next 2447
2021-08-09 03:52:38.098293: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1400 of size 256 by op assert_greater_equal_7/Assert/AssertGuard/pivot_t/_200 action_count 94195455584315 step 0 next 1721
2021-08-09 03:52:38.098314: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1500 of size 256 by op assert_less_equal_7/Assert/AssertGuard/pivot_f/_209 action_count 94195455584316 step 0 next 1588
2021-08-09 03:52:38.098335: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1600 of size 256 by op assert_less_equal_7/Assert/AssertGuard/pivot_t/_210 action_count 94195455584317 step 0 next 2259
2021-08-09 03:52:38.098351: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1700 of size 256 by op assert_greater_equal_8/Assert/AssertGuard/pivot_f/_227 action_count 94195455584318 step 0 next 1805
2021-08-09 03:52:38.098367: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1800 of size 256 by op Sub action_count 94195129465111 step 0 next 2174
2021-08-09 03:52:38.098386: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1900 of size 256 by op Sub action_count 94195129465112 step 0 next 2091
2021-08-09 03:52:38.098406: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1a00 of size 1024 by op Fill action_count 94195480111064 step 0 next 1925
2021-08-09 03:52:38.098422: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c1e00 of size 1024 by op Fill action_count 94195480111065 step 0 next 1834
2021-08-09 03:52:38.098442: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c2200 of size 1024 by op Fill action_count 94195480111066 step 0 next 1061
2021-08-09 03:52:38.098460: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c2600 of size 1024 by op Fill action_count 94195480111067 step 0 next 1048
2021-08-09 03:52:38.098476: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c2a00 of size 1024 by op Fill action_count 94195480111068 step 0 next 284
2021-08-09 03:52:38.098495: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c2e00 of size 1024 by op Fill action_count 94195480111078 step 0 next 2346
2021-08-09 03:52:38.098515: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c3200 of size 1024 by op Fill action_count 94195480111079 step 0 next 2238
2021-08-09 03:52:38.098545: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c3600 of size 1024 by op Fill action_count 94195480111080 step 0 next 1526
2021-08-09 03:52:38.098565: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c3a00 of size 1024 by op Fill action_count 94195480111081 step 0 next 1710
2021-08-09 03:52:38.098584: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c3e00 of size 1024 by op Fill action_count 94195480111082 step 0 next 644
2021-08-09 03:52:38.098603: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c4200 of size 1024 by op Fill action_count 94195480111090 step 0 next 1791
2021-08-09 03:52:38.098619: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c4600 of size 1024 by op Fill action_count 94195480111091 step 0 next 1623
2021-08-09 03:52:38.098640: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c4a00 of size 1024 by op Fill action_count 94195480111092 step 0 next 2074
2021-08-09 03:52:38.098658: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c4e00 of size 1024 by op Fill action_count 94195480111093 step 0 next 2059
2021-08-09 03:52:38.098677: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5200 of size 1024 by op Fill action_count 94195480111094 step 0 next 2095
2021-08-09 03:52:38.098694: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5600 of size 256 by op Fill action_count 94195480111104 step 0 next 544
2021-08-09 03:52:38.098713: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5700 of size 256 by op Fill action_count 94195480111105 step 0 next 1572
2021-08-09 03:52:38.098732: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5800 of size 256 by op Fill action_count 94195480111106 step 0 next 18
2021-08-09 03:52:38.098751: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5900 of size 256 by op Fill action_count 94195480111107 step 0 next 578
2021-08-09 03:52:38.098767: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5a00 of size 256 by op Fill action_count 94195480111108 step 0 next 1311
2021-08-09 03:52:38.098782: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5b00 of size 256 by op Fill action_count 94195480111109 step 0 next 1385
2021-08-09 03:52:38.098801: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5c00 of size 256 by op Fill action_count 94195480111110 step 0 next 1664
2021-08-09 03:52:38.098820: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5d00 of size 256 by op Fill action_count 94195480111111 step 0 next 596
2021-08-09 03:52:38.098852: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c5e00 of size 1024 by op Add action_count 94195480111099 step 0 next 2190
2021-08-09 03:52:38.098872: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6200 of size 256 by op Fill action_count 94195480111112 step 0 next 2621
2021-08-09 03:52:38.098890: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6300 of size 256 by op Fill action_count 94195480111113 step 0 next 2146
2021-08-09 03:52:38.098909: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6400 of size 256 by op Fill action_count 94195480111114 step 0 next 2195
2021-08-09 03:52:38.098927: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6500 of size 256 by op Fill action_count 94195480111115 step 0 next 1111
2021-08-09 03:52:38.098945: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6600 of size 256 by op Fill action_count 94195480111116 step 0 next 1789
2021-08-09 03:52:38.098963: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6700 of size 256 by op Fill action_count 94195480111117 step 0 next 1301
2021-08-09 03:52:38.098981: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6800 of size 256 by op Fill action_count 94195480111118 step 0 next 2286
2021-08-09 03:52:38.098998: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6900 of size 256 by op Fill action_count 94195480111119 step 0 next 1706
2021-08-09 03:52:38.099016: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6a00 of size 256 by op Fill action_count 94195480111120 step 0 next 858
2021-08-09 03:52:38.099033: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6b00 of size 256 by op Fill action_count 94195480111121 step 0 next 2147
2021-08-09 03:52:38.099048: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6c00 of size 256 by op Fill action_count 94195480111122 step 0 next 2592
2021-08-09 03:52:38.099063: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6d00 of size 256 by op Fill action_count 94195480111123 step 0 next 826
2021-08-09 03:52:38.099078: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6e00 of size 256 by op Fill action_count 94195480111124 step 0 next 449
2021-08-09 03:52:38.099093: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c6f00 of size 256 by op Fill action_count 94195480111125 step 0 next 767
2021-08-09 03:52:38.099107: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7000 of size 256 by op Fill action_count 94195480111126 step 0 next 347
2021-08-09 03:52:38.099122: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7100 of size 256 by op Fill action_count 94195480111127 step 0 next 524
2021-08-09 03:52:38.099140: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7200 of size 256 by op Fill action_count 94195480111128 step 0 next 653
2021-08-09 03:52:38.099158: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7300 of size 256 by op Fill action_count 94195480111129 step 0 next 764
2021-08-09 03:52:38.099175: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7400 of size 256 by op Fill action_count 94195480111130 step 0 next 938
2021-08-09 03:52:38.099193: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7500 of size 256 by op Fill action_count 94195480111131 step 0 next 999
2021-08-09 03:52:38.099211: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7600 of size 256 by op Fill action_count 94195480111132 step 0 next 916
2021-08-09 03:52:38.099229: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7700 of size 256 by op Fill action_count 94195480111133 step 0 next 2140
2021-08-09 03:52:38.099246: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7800 of size 256 by op Fill action_count 94195480111134 step 0 next 523
2021-08-09 03:52:38.099264: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7900 of size 256 by op Fill action_count 94195480111135 step 0 next 1146
2021-08-09 03:52:38.099282: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7a00 of size 256 by op Fill action_count 94195480111136 step 0 next 952
2021-08-09 03:52:38.099300: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7b00 of size 256 by op Fill action_count 94195480111137 step 0 next 423
2021-08-09 03:52:38.099318: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7c00 of size 256 by op Fill action_count 94195480111138 step 0 next 2423
2021-08-09 03:52:38.099336: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7d00 of size 256 by op Fill action_count 94195480111139 step 0 next 250
2021-08-09 03:52:38.099354: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7e00 of size 256 by op Fill action_count 94195480111140 step 0 next 859
2021-08-09 03:52:38.099373: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c7f00 of size 256 by op Fill action_count 94195480111141 step 0 next 1186
2021-08-09 03:52:38.099391: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8000 of size 256 by op Fill action_count 94195480111142 step 0 next 1299
2021-08-09 03:52:38.099408: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8100 of size 256 by op Fill action_count 94195480111143 step 0 next 2622
2021-08-09 03:52:38.099427: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8200 of size 256 by op Fill action_count 94195480111144 step 0 next 2076
2021-08-09 03:52:38.099446: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8300 of size 512 by op Fill action_count 94195480111145 step 0 next 671
2021-08-09 03:52:38.099461: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8500 of size 512 by op Fill action_count 94195480111146 step 0 next 1923
2021-08-09 03:52:38.099479: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8700 of size 512 by op Fill action_count 94195480111147 step 0 next 2044
2021-08-09 03:52:38.099497: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8900 of size 512 by op Fill action_count 94195480111148 step 0 next 1859
2021-08-09 03:52:38.099515: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8b00 of size 256 by op Fill action_count 94195480111149 step 0 next 603
2021-08-09 03:52:38.099535: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8c00 of size 256 by op Fill action_count 94195480111150 step 0 next 447
2021-08-09 03:52:38.099550: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8d00 of size 256 by op Fill action_count 94195480111151 step 0 next 362
2021-08-09 03:52:38.099565: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8e00 of size 256 by op Fill action_count 94195480111152 step 0 next 2367
2021-08-09 03:52:38.099583: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c8f00 of size 256 by op Fill action_count 94195480111153 step 0 next 467
2021-08-09 03:52:38.099601: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c9000 of size 256 by op Fill action_count 94195480111154 step 0 next 1100
2021-08-09 03:52:38.099619: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c9100 of size 256 by op AssignVariableOp action_count 94195480111155 step 0 next 91
2021-08-09 03:52:38.099637: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c9200 of size 256 by op Fill action_count 94195480111156 step 0 next 236
2021-08-09 03:52:38.099655: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c9300 of size 1536 by op Fill action_count 94195480111157 step 0 next 857
2021-08-09 03:52:38.099673: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c9900 of size 512 by op Fill action_count 94195480111158 step 0 next 1779
2021-08-09 03:52:38.099691: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c9b00 of size 512 by op Fill action_count 94195480111159 step 0 next 879
2021-08-09 03:52:38.099708: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c9d00 of size 512 by op Fill action_count 94195480111160 step 0 next 1191
2021-08-09 03:52:38.099723: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47c9f00 of size 512 by op Fill action_count 94195480111162 step 0 next 2373
2021-08-09 03:52:38.099742: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ca100 of size 512 by op Fill action_count 94195480111163 step 0 next 212
2021-08-09 03:52:38.099760: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ca300 of size 512 by op Fill action_count 94195480111164 step 0 next 27
2021-08-09 03:52:38.099778: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ca500 of size 1024 by op Fill action_count 94195480111166 step 0 next 2453
2021-08-09 03:52:38.099796: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ca900 of size 1024 by op Fill action_count 94195480111167 step 0 next 120
2021-08-09 03:52:38.099814: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cad00 of size 1024 by op Fill action_count 94195480111168 step 0 next 398
2021-08-09 03:52:38.099833: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cb100 of size 1024 by op Fill action_count 94195480111170 step 0 next 394
2021-08-09 03:52:38.099851: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cb500 of size 1024 by op Fill action_count 94195480111171 step 0 next 1774
2021-08-09 03:52:38.099869: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cb900 of size 1024 by op Fill action_count 94195480111172 step 0 next 881
2021-08-09 03:52:38.099887: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cbd00 of size 2048 by op Fill action_count 94195480111174 step 0 next 1019
2021-08-09 03:52:38.099905: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cc500 of size 2048 by op Fill action_count 94195480111175 step 0 next 1787
2021-08-09 03:52:38.099923: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ccd00 of size 2048 by op Fill action_count 94195480111176 step 0 next 768
2021-08-09 03:52:38.099941: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cd500 of size 2048 by op Fill action_count 94195480111178 step 0 next 756
2021-08-09 03:52:38.099959: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cdd00 of size 2048 by op Fill action_count 94195480111179 step 0 next 1265
2021-08-09 03:52:38.099978: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ce500 of size 2048 by op Fill action_count 94195480111180 step 0 next 724
2021-08-09 03:52:38.099997: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ced00 of size 2048 by op Fill action_count 94195480111182 step 0 next 2571
2021-08-09 03:52:38.100015: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cf500 of size 2048 by op Fill action_count 94195480111183 step 0 next 562
2021-08-09 03:52:38.100033: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47cfd00 of size 2048 by op Fill action_count 94195480111184 step 0 next 1038
2021-08-09 03:52:38.100051: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d0500 of size 2048 by op Fill action_count 94195480111186 step 0 next 145
2021-08-09 03:52:38.100069: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d0d00 of size 2048 by op Fill action_count 94195480111187 step 0 next 1627
2021-08-09 03:52:38.100086: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d1500 of size 2048 by op Fill action_count 94195480111188 step 0 next 84
2021-08-09 03:52:38.100101: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d1d00 of size 4096 by op Fill action_count 94195480111190 step 0 next 882
2021-08-09 03:52:38.100116: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d2d00 of size 4096 by op Fill action_count 94195480111191 step 0 next 529
2021-08-09 03:52:38.100131: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d3d00 of size 4096 by op Fill action_count 94195480111192 step 0 next 944
2021-08-09 03:52:38.100146: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d4d00 of size 4096 by op Fill action_count 94195480111194 step 0 next 1955
2021-08-09 03:52:38.100161: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d5d00 of size 4096 by op Fill action_count 94195480111195 step 0 next 122
2021-08-09 03:52:38.100176: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d6d00 of size 4096 by op Fill action_count 94195480111196 step 0 next 1974
2021-08-09 03:52:38.100194: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d7d00 of size 2048 by op Fill action_count 94195480111198 step 0 next 1647
2021-08-09 03:52:38.100212: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d8500 of size 2048 by op Fill action_count 94195480111199 step 0 next 1830
2021-08-09 03:52:38.100230: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d8d00 of size 2048 by op Fill action_count 94195480111200 step 0 next 1244
2021-08-09 03:52:38.100248: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d9500 of size 2048 by op Fill action_count 94195480111202 step 0 next 967
2021-08-09 03:52:38.100266: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47d9d00 of size 2048 by op Fill action_count 94195480111203 step 0 next 1488
2021-08-09 03:52:38.100285: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47da500 of size 2048 by op Fill action_count 94195480111204 step 0 next 561
2021-08-09 03:52:38.100303: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47dad00 of size 2048 by op Fill action_count 94195480111206 step 0 next 803
2021-08-09 03:52:38.100321: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47db500 of size 2048 by op Fill action_count 94195480111207 step 0 next 281
2021-08-09 03:52:38.100339: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47dbd00 of size 2048 by op Fill action_count 94195480111208 step 0 next 1849
2021-08-09 03:52:38.100357: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47dc500 of size 2048 by op Fill action_count 94195480111210 step 0 next 1810
2021-08-09 03:52:38.100375: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47dcd00 of size 2048 by op Fill action_count 94195480111211 step 0 next 535
2021-08-09 03:52:38.100394: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47dd500 of size 2048 by op Fill action_count 94195480111212 step 0 next 1798
2021-08-09 03:52:38.100411: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ddd00 of size 2048 by op Fill action_count 94195480111214 step 0 next 2084
2021-08-09 03:52:38.100425: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47de500 of size 2304 by op Fill action_count 94195480111215 step 0 next 2168
2021-08-09 03:52:38.100440: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47dee00 of size 256 by op Sub action_count 94195129465161 step 0 next 2177
2021-08-09 03:52:38.100455: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47def00 of size 256 by op Sub action_count 94195129465162 step 0 next 2086
2021-08-09 03:52:38.100470: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47df000 of size 3584 by op Fill action_count 94195455013692 step 0 next 415
2021-08-09 03:52:38.100488: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47dfe00 of size 4096 by op Fill action_count 94195455013694 step 0 next 1246
2021-08-09 03:52:38.100507: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e0e00 of size 256 by op Sub action_count 94195129465199 step 0 next 989
2021-08-09 03:52:38.100561: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e0f00 of size 256 by op Sub action_count 94195129465200 step 0 next 2109
2021-08-09 03:52:38.100592: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e1000 of size 1792 by op Fill action_count 94195455013712 step 0 next 2066
2021-08-09 03:52:38.100604: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e1700 of size 1792 by op Fill action_count 94195455013714 step 0 next 2933
2021-08-09 03:52:38.100613: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e1e00 of size 1792 by op Fill action_count 94195455013715 step 0 next 1104
2021-08-09 03:52:38.100623: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e2500 of size 2816 by op Fill action_count 94195455013716 step 0 next 2454
2021-08-09 03:52:38.100632: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e3000 of size 256 by op Sub action_count 94195365012667 step 0 next 1811
2021-08-09 03:52:38.100642: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e3100 of size 256 by op Sub action_count 94195365012668 step 0 next 1998
2021-08-09 03:52:38.100651: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e3200 of size 1792 by op Fill action_count 94195455013781 step 0 next 602
2021-08-09 03:52:38.100660: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e3900 of size 1792 by op Fill action_count 94195455013782 step 0 next 214
2021-08-09 03:52:38.100670: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e4000 of size 1792 by op Fill action_count 94195455013784 step 0 next 1253
2021-08-09 03:52:38.100679: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e4700 of size 1792 by op Fill action_count 94195455013785 step 0 next 2398
2021-08-09 03:52:38.100688: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e4e00 of size 1792 by op Fill action_count 94195455013786 step 0 next 1972
2021-08-09 03:52:38.100697: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e5500 of size 3328 by op Fill action_count 94195455013788 step 0 next 2285
2021-08-09 03:52:38.100706: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e6200 of size 256 by op Sub action_count 94195339127966 step 0 next 1832
2021-08-09 03:52:38.100716: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e6300 of size 1024 by op Fill action_count 94195455013408 step 0 next 333
2021-08-09 03:52:38.100725: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e6700 of size 256 by op Sub action_count 94195339127967 step 0 next 1754
2021-08-09 03:52:38.100734: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e6800 of size 2048 by op Fill action_count 94195455013454 step 0 next 721
2021-08-09 03:52:38.100744: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e7000 of size 256 by op Sub action_count 94195339127980 step 0 next 853
2021-08-09 03:52:38.100753: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e7100 of size 1792 by op Fill action_count 94195455013558 step 0 next 2745
2021-08-09 03:52:38.100762: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e7800 of size 1792 by op Fill action_count 94195455013559 step 0 next 890
2021-08-09 03:52:38.100771: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e7f00 of size 2048 by op Fill action_count 94195455013560 step 0 next 2358
2021-08-09 03:52:38.100780: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e8700 of size 256 by op Sub action_count 94195339127995 step 0 next 845
2021-08-09 03:52:38.100789: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e8800 of size 2048 by op Fill action_count 94195480110908 step 0 next 2413
2021-08-09 03:52:38.100798: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e9000 of size 2048 by op Fill action_count 94195480110916 step 0 next 1780
2021-08-09 03:52:38.100808: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47e9800 of size 2048 by op Fill action_count 94195480110917 step 0 next 61
2021-08-09 03:52:38.100817: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ea000 of size 2048 by op Fill action_count 94195480110918 step 0 next 865
2021-08-09 03:52:38.100826: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ea800 of size 2048 by op Fill action_count 94195480110919 step 0 next 1054
2021-08-09 03:52:38.100835: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47eb000 of size 2048 by op Fill action_count 94195480110920 step 0 next 510
2021-08-09 03:52:38.100845: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47eb800 of size 4096 by op Fill action_count 94195480110928 step 0 next 1992
2021-08-09 03:52:38.100854: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ec800 of size 4096 by op Fill action_count 94195480110929 step 0 next 1912
2021-08-09 03:52:38.100863: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ed800 of size 4096 by op Fill action_count 94195480110930 step 0 next 1156
2021-08-09 03:52:38.100892: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ee800 of size 4096 by op Fill action_count 94195480110931 step 0 next 167
2021-08-09 03:52:38.100902: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47ef800 of size 4096 by op Fill action_count 94195480110932 step 0 next 1745
2021-08-09 03:52:38.100911: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f0800 of size 4096 by op Fill action_count 94195480110940 step 0 next 805
2021-08-09 03:52:38.100920: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f1800 of size 4096 by op Fill action_count 94195480110941 step 0 next 105
2021-08-09 03:52:38.100929: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f2800 of size 4096 by op Fill action_count 94195480110942 step 0 next 1729
2021-08-09 03:52:38.100938: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f3800 of size 4096 by op Fill action_count 94195480110943 step 0 next 1659
2021-08-09 03:52:38.100947: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f4800 of size 4096 by op Fill action_count 94195480110944 step 0 next 1107
2021-08-09 03:52:38.100956: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f5800 of size 2048 by op Fill action_count 94195480110952 step 0 next 1086
2021-08-09 03:52:38.100965: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f6000 of size 2048 by op Fill action_count 94195480110953 step 0 next 353
2021-08-09 03:52:38.100974: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f6800 of size 2048 by op Fill action_count 94195480110954 step 0 next 556
2021-08-09 03:52:38.100983: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f7000 of size 2048 by op Fill action_count 94195480110955 step 0 next 1860
2021-08-09 03:52:38.100992: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f7800 of size 3840 by op Fill action_count 94195480110956 step 0 next 1247
2021-08-09 03:52:38.101001: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f8700 of size 256 by op Sub action_count 94195129465225 step 0 next 2092
2021-08-09 03:52:38.101010: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f8800 of size 256 by op Sub action_count 94195129465226 step 0 next 1900
2021-08-09 03:52:38.101021: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f8900 of size 1792 by op Fill action_count 94195455013752 step 0 next 2603
2021-08-09 03:52:38.101031: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f9000 of size 1792 by op Fill action_count 94195455013753 step 0 next 1335
2021-08-09 03:52:38.101041: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f9700 of size 1792 by op Fill action_count 94195455013754 step 0 next 1537
2021-08-09 03:52:38.101051: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47f9e00 of size 1792 by op Fill action_count 94195455013756 step 0 next 182
2021-08-09 03:52:38.101061: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47fa500 of size 2304 by op Fill action_count 94195455013757 step 0 next 946
2021-08-09 03:52:38.101071: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47fae00 of size 256 by op Sub action_count 94195152301760 step 0 next 442
2021-08-09 03:52:38.101081: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47faf00 of size 1280 by op Fill action_count 94195455013412 step 0 next 274
2021-08-09 03:52:38.101091: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47fb400 of size 256 by op Sub action_count 94195152301802 step 0 next 966
2021-08-09 03:52:38.101101: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47fb500 of size 256 by op Sub action_count 94195152301803 step 0 next 110
2021-08-09 03:52:38.101317: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47fb600 of size 256 by op Sub action_count 94195152301828 step 0 next 1865
2021-08-09 03:52:38.101375: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae47fb700 of size 7012352 by op Add action_count 94195455013529 step 0 next 781
2021-08-09 03:52:38.101423: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae4eab700 of size 24661504 by op Fill action_count 94195455013697 step 0 next 614
2021-08-09 03:52:38.101475: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6630500 of size 256 by op Sub action_count 94195273335853 step 0 next 630
2021-08-09 03:52:38.101507: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6630600 of size 256 by op Sub action_count 94195273335854 step 0 next 1017
2021-08-09 03:52:38.101661: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6630700 of size 3584 by op Fill action_count 94195455013695 step 0 next 2225
2021-08-09 03:52:38.101692: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6631500 of size 4096 by op Fill action_count 94195455013696 step 0 next 559
2021-08-09 03:52:38.101715: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6632500 of size 256 by op Sub action_count 94195273335867 step 0 next 918
2021-08-09 03:52:38.101739: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6632600 of size 256 by op Sub action_count 94195273335868 step 0 next 780
2021-08-09 03:52:38.101765: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6632700 of size 2048 by op Fill action_count 94195455013698 step 0 next 2417
2021-08-09 03:52:38.101793: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6632f00 of size 2048 by op Fill action_count 94195455013699 step 0 next 1044
2021-08-09 03:52:38.101819: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6633700 of size 3584 by op Fill action_count 94195455013700 step 0 next 1815
2021-08-09 03:52:38.101846: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6634500 of size 256 by op Sub action_count 94195273335881 step 0 next 827
2021-08-09 03:52:38.101871: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6634600 of size 256 by op Sub action_count 94195273335882 step 0 next 1995
2021-08-09 03:52:38.101895: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6634700 of size 1024 by op Fill action_count 94195467563084 step 0 next 1248
2021-08-09 03:52:38.101920: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6634b00 of size 1024 by op Fill action_count 94195467563085 step 0 next 1328
2021-08-09 03:52:38.101946: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6634f00 of size 1024 by op Fill action_count 94195467563086 step 0 next 873
2021-08-09 03:52:38.101984: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6635300 of size 1024 by op Fill action_count 94195467563087 step 0 next 384
2021-08-09 03:52:38.102011: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6635700 of size 1024 by op Fill action_count 94195467563095 step 0 next 1325
2021-08-09 03:52:38.102037: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6635b00 of size 1024 by op Fill action_count 94195467563096 step 0 next 537
2021-08-09 03:52:38.102062: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6635f00 of size 1024 by op Fill action_count 94195467563097 step 0 next 2634
2021-08-09 03:52:38.102087: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6636300 of size 1024 by op Fill action_count 94195467563098 step 0 next 1950
2021-08-09 03:52:38.102114: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6636700 of size 1024 by op Fill action_count 94195467563099 step 0 next 2660
2021-08-09 03:52:38.102143: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6636b00 of size 256 by op AssignVariableOp action_count 94195480053238 step 0 next 1239
2021-08-09 03:52:38.102171: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6636c00 of size 256 by op AssignVariableOp action_count 94195480053240 step 0 next 478
2021-08-09 03:52:38.102197: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6636d00 of size 256 by op Sub action_count 94195480110994 step 0 next 821
2021-08-09 03:52:38.102240: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6636e00 of size 256 by op AssignAddVariableOp_12 action_count 94195480109429 step 17368542405085625385 next 778
2021-08-09 03:52:38.102270: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6636f00 of size 256 by op Sub action_count 94195480110995 step 0 next 1693
2021-08-09 03:52:38.102299: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637000 of size 256 by op AssignVariableOp action_count 94195480053234 step 0 next 1852
2021-08-09 03:52:38.102327: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637100 of size 256 by op AssignVariableOp action_count 94195480053236 step 0 next 1129
2021-08-09 03:52:38.102352: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637200 of size 256 by op Sub action_count 94195480111032 step 0 next 1961
2021-08-09 03:52:38.102378: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637300 of size 1024 by op Add action_count 94195467563104 step 0 next 2418
2021-08-09 03:52:38.102405: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637700 of size 256 by op Sub action_count 94195480111033 step 0 next 2315
2021-08-09 03:52:38.102429: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637800 of size 256 by op AssignVariableOp action_count 94195480053226 step 0 next 1180
2021-08-09 03:52:38.102466: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637900 of size 256 by op Sub action_count 94195480111070 step 0 next 710
2021-08-09 03:52:38.102495: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637a00 of size 256 by op AssignAddVariableOp_10 action_count 94195480109361 step 17368542405085625385 next 2555
2021-08-09 03:52:38.102540: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637b00 of size 1024 by op Fill action_count 94195480110850 step 0 next 1888
2021-08-09 03:52:38.102570: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6637f00 of size 1280 by op Fill action_count 94195480110851 step 0 next 2317
2021-08-09 03:52:38.102595: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638400 of size 256 by op AssignAddVariableOp_52 action_count 94195480053815 step 12509324397843644032 next 153
2021-08-09 03:52:38.102624: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638500 of size 512 by op Fill action_count 94195480110836 step 0 next 835
2021-08-09 03:52:38.102652: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638700 of size 768 by op Fill action_count 94195480110837 step 0 next 958
2021-08-09 03:52:38.102683: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638a00 of size 256 by op AssignAddVariableOp_20 action_count 94195480109433 step 17368542405085625385 next 1113
2021-08-09 03:52:38.102710: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638b00 of size 256 by op AssignAddVariableOp_18 action_count 94195480109373 step 17368542405085625385 next 292
2021-08-09 03:52:38.102742: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638c00 of size 256 by op AssignAddVariableOp_28 action_count 94195480109437 step 17368542405085625385 next 610
2021-08-09 03:52:38.102778: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638d00 of size 256 by op AssignVariableOp action_count 94195480053198 step 0 next 226
2021-08-09 03:52:38.102804: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638e00 of size 256 by op AssignVariableOp action_count 94195480053204 step 0 next 1951
2021-08-09 03:52:38.102844: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6638f00 of size 256 by op AssignAddVariableOp_11 action_count 94195480109363 step 17368542405085625385 next 2366
2021-08-09 03:52:38.102873: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639000 of size 256 by op Sub action_count 94195480111071 step 0 next 2282
2021-08-09 03:52:38.102898: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639100 of size 256 by op AssignVariableOp action_count 94195479483519 step 0 next 299
2021-08-09 03:52:38.102926: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639200 of size 256 by op AssignVariableOp action_count 94195480053200 step 0 next 1839
2021-08-09 03:52:38.102955: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639300 of size 256 by op Fill action_count 94195480111102 step 0 next 1398
2021-08-09 03:52:38.102990: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639400 of size 256 by op ConstantFolding/truediv_recip action_count 94195468133239 step 0 next 2053
2021-08-09 03:52:38.103020: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639500 of size 256 by op add_16/y action_count 94195468133240 step 0 next 2983
2021-08-09 03:52:38.103048: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639600 of size 256 by op Sum_1 action_count 94195468133241 step 0 next 1948
2021-08-09 03:52:38.103075: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639700 of size 256 by op truediv/y action_count 94195468133243 step 0 next 1020
2021-08-09 03:52:38.103099: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639800 of size 256 by op binary_ce_dice/add_1/y action_count 94195468133244 step 0 next 1875
2021-08-09 03:52:38.103134: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639900 of size 256 by op AssignVariableOp action_count 94195480053248 step 0 next 436
2021-08-09 03:52:38.103163: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639a00 of size 256 by op AssignVariableOp action_count 94195480053216 step 0 next 1527
2021-08-09 03:52:38.103186: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639b00 of size 256 by op Fill action_count 94195480111103 step 0 next 1241
2021-08-09 03:52:38.103210: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639c00 of size 256 by op Fill action_count 94195467563161 step 0 next 1855
2021-08-09 03:52:38.103236: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6639d00 of size 1536 by op Fill action_count 94195467563162 step 0 next 487
2021-08-09 03:52:38.103263: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663a300 of size 512 by op Fill action_count 94195467563163 step 0 next 2496
2021-08-09 03:52:38.103296: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663a500 of size 512 by op Fill action_count 94195467563164 step 0 next 969
2021-08-09 03:52:38.103324: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663a700 of size 512 by op Fill action_count 94195467563165 step 0 next 1968
2021-08-09 03:52:38.103346: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663a900 of size 512 by op Fill action_count 94195467563167 step 0 next 1922
2021-08-09 03:52:38.103371: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663ab00 of size 512 by op Fill action_count 94195467563168 step 0 next 698
2021-08-09 03:52:38.103397: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663ad00 of size 512 by op Fill action_count 94195467563169 step 0 next 1397
2021-08-09 03:52:38.103423: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663af00 of size 1024 by op Fill action_count 94195467563171 step 0 next 1202
2021-08-09 03:52:38.103450: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663b300 of size 1024 by op Fill action_count 94195467563172 step 0 next 2420
2021-08-09 03:52:38.103484: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663b700 of size 1024 by op Fill action_count 94195467563173 step 0 next 194
2021-08-09 03:52:38.103512: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663bb00 of size 1024 by op Fill action_count 94195467563175 step 0 next 2276
2021-08-09 03:52:38.103546: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663bf00 of size 1024 by op Fill action_count 94195467563176 step 0 next 2593
2021-08-09 03:52:38.103571: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663c300 of size 1024 by op Fill action_count 94195467563177 step 0 next 1395
2021-08-09 03:52:38.103593: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663c700 of size 1792 by op Fill action_count 94195467563179 step 0 next 217
2021-08-09 03:52:38.103618: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663ce00 of size 1792 by op Fill action_count 94195467563180 step 0 next 804
2021-08-09 03:52:38.103664: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663d500 of size 1792 by op Fill action_count 94195467563181 step 0 next 1966
2021-08-09 03:52:38.103693: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663dc00 of size 1792 by op Fill action_count 94195467563183 step 0 next 1491
2021-08-09 03:52:38.103719: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663e300 of size 256 by op Sub action_count 94195273335919 step 0 next 1963
2021-08-09 03:52:38.103746: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663e400 of size 256 by op Sub action_count 94195273335920 step 0 next 424
2021-08-09 03:52:38.103774: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663e500 of size 3584 by op Fill action_count 94195455013765 step 0 next 963
2021-08-09 03:52:38.103800: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae663f300 of size 6656 by op Fill action_count 94195455013766 step 0 next 867
2021-08-09 03:52:38.103835: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6640d00 of size 256 by op Sub action_count 94195273335957 step 0 next 920
2021-08-09 03:52:38.103864: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6640e00 of size 256 by op Sub action_count 94195273335958 step 0 next 1318
2021-08-09 03:52:38.103891: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6640f00 of size 1792 by op Fill action_count 94195455013561 step 0 next 85
2021-08-09 03:52:38.103917: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6641600 of size 1792 by op Fill action_count 94195455013562 step 0 next 1266
2021-08-09 03:52:38.103943: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6641d00 of size 1024 by op Fill action_count 94195455013570 step 0 next 1684
2021-08-09 03:52:38.103968: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6642100 of size 1024 by op Fill action_count 94195455013571 step 0 next 211
2021-08-09 03:52:38.104005: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6642500 of size 256 by op Sub action_count 94195273335983 step 0 next 841
2021-08-09 03:52:38.104031: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6642600 of size 256 by op Sub action_count 94195273335984 step 0 next 1782
2021-08-09 03:52:38.104062: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6642700 of size 8192000 by op Tile_90/_0__cf__3862 action_count 94195410013342 step 0 next 1480
2021-08-09 03:52:38.104090: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6e12700 of size 1099264 by op Fill action_count 94195455013681 step 0 next 2749
2021-08-09 03:52:38.104118: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6f1ed00 of size 549632 by op Fill action_count 94195455013747 step 0 next 1434
2021-08-09 03:52:38.104146: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae6fa5000 of size 1055488 by op Fill action_count 94195455013795 step 0 next 2865
2021-08-09 03:52:38.104181: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae70a6b00 of size 256 by op AssignAddVariableOp_52 action_count 94195410014265 step 11906430663947522641 next 2684
2021-08-09 03:52:38.104211: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae70a6c00 of size 8192000 by op Tile_90/_0__cf__3864 action_count 94195412081363 step 0 next 2864
2021-08-09 03:52:38.104238: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae7876c00 of size 8792832 by op Fill action_count 94195467563190 step 0 next 2262
2021-08-09 03:52:38.104266: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae80d9700 of size 8792832 by op Add action_count 94195467562984 step 0 next 2484
2021-08-09 03:52:38.104292: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae893c200 of size 5775360 by op Add action_count 94195467563008 step 0 next 888
2021-08-09 03:52:38.104329: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae8ebe200 of size 3297536 by op Add action_count 94195467563056 step 0 next 308
2021-08-09 03:52:38.104359: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae91e3300 of size 3714816 by op Add action_count 94195467563044 step 0 next 1489
2021-08-09 03:52:38.104385: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae956e200 of size 7012352 by op Add action_count 94195467563032 step 0 next 2216
2021-08-09 03:52:38.104411: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae9c1e200 of size 2930944 by op Fill action_count 94195467563182 step 0 next 1615
2021-08-09 03:52:38.104449: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ae9ee9b00 of size 5318400 by op Add action_count 94195467563068 step 0 next 2220
2021-08-09 03:52:38.104478: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aea3fc200 of size 14024704 by op Add action_count 94195467562996 step 0 next 1230
2021-08-09 03:52:38.104504: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aeb15c200 of size 14024704 by op Fill action_count 94195467563194 step 0 next 1924
2021-08-09 03:52:38.104538: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aebebc200 of size 5775360 by op Fill action_count 94195467563198 step 0 next 875
2021-08-09 03:52:38.104577: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aec43e200 of size 3145728 by op Fill action_count 94195467563202 step 0 next 1655
2021-08-09 03:52:38.104607: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aec73e200 of size 7012352 by op Fill action_count 94195467563206 step 0 next 940
2021-08-09 03:52:38.104635: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aecdee200 of size 3297536 by op Fill action_count 94195467563210 step 0 next 325
2021-08-09 03:52:38.104661: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aed113300 of size 2198272 by op Fill action_count 94195467563214 step 0 next 1562
2021-08-09 03:52:38.104695: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aed32be00 of size 2930944 by op Fill action_count 94195467563218 step 0 next 2478
2021-08-09 03:52:38.104732: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aed5f7700 of size 824576 by op Fill action_count 94195467563222 step 0 next 1762
2021-08-09 03:52:38.104761: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aed6c0c00 of size 274944 by op Fill action_count 94195467563240 step 0 next 990
2021-08-09 03:52:38.104787: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aed703e00 of size 549632 by op Fill action_count 94195467563244 step 0 next 2485
2021-08-09 03:52:38.104813: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aed78a100 of size 1099264 by op Fill action_count 94195467563248 step 0 next 646
2021-08-09 03:52:38.104838: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aed896700 of size 2198272 by op Fill action_count 94195467563252 step 0 next 359
2021-08-09 03:52:38.104876: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aedaaf200 of size 4396544 by op Fill action_count 94195467563256 step 0 next 480
2021-08-09 03:52:38.104902: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aedee0800 of size 8792832 by op Fill action_count 94195467563260 step 0 next 1417
2021-08-09 03:52:38.104929: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aee743300 of size 14024704 by op Fill action_count 94195467563264 step 0 next 2389
2021-08-09 03:52:38.104957: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aef4a3300 of size 5775360 by op Fill action_count 94195467563268 step 0 next 1738
2021-08-09 03:52:38.104985: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aefa25300 of size 3145728 by op Fill action_count 94195467563272 step 0 next 207
2021-08-09 03:52:38.105021: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3aefd25300 of size 7012352 by op Fill action_count 94195467563276 step 0 next 1300
2021-08-09 03:52:38.105048: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af03d5300 of size 3297536 by op Fill action_count 94195467563280 step 0 next 2101
2021-08-09 03:52:38.105074: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af06fa400 of size 2198272 by op Fill action_count 94195467563284 step 0 next 2381
2021-08-09 03:52:38.105102: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0912f00 of size 2930944 by op Fill action_count 94195467563288 step 0 next 790
2021-08-09 03:52:38.105128: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0bde800 of size 824576 by op Fill action_count 94195467563292 step 0 next 759
2021-08-09 03:52:38.105165: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0ca7d00 of size 549632 by op Fill action_count 94195467563296 step 0 next 2598
2021-08-09 03:52:38.105196: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0d2e000 of size 425984 by op gradient_tape/binary_ce_dice/logistic_loss/zeros_like action_count 94195467563312 step 0 next 1368
2021-08-09 03:52:38.105224: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0d96000 of size 425984 by op Tile_36 action_count 94195467563317 step 0 next 1772
2021-08-09 03:52:38.105253: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0dfe000 of size 425984 by op Tile_54 action_count 94195467563318 step 0 next 2232
2021-08-09 03:52:38.105289: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0e66000 of size 425984 by op Tile_72 action_count 94195467563319 step 0 next 1208
2021-08-09 03:52:38.105321: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0ece000 of size 425984 by op Tile_93 action_count 94195467563321 step 0 next 704
2021-08-09 03:52:38.105350: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0f36000 of size 425984 by op Tile_18 action_count 94195467563322 step 0 next 534
2021-08-09 03:52:38.105377: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af0f9e000 of size 425984 by op Tile action_count 94195467563323 step 0 next 216
2021-08-09 03:52:38.105416: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af1006000 of size 425984 by op gradient_tape/binary_ce_dice/logistic_loss/sub/Neg/_0__cf__4148 action_count 94195467563478 step 0 next 1578
2021-08-09 03:52:38.105447: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af106e000 of size 425984 by op gradient_tape/binary_ce_dice/weighted_loss/Tile_1/_1__cf__4149 action_count 94195467563479 step 0 next 1495
2021-08-09 03:52:38.105474: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af10d6000 of size 425984 by op binary_ce_dice/logistic_loss/zeros_like action_count 94195468133242 step 0 next 1119
2021-08-09 03:52:38.105501: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af113e000 of size 425984 by op Tile action_count 94195468133246 step 0 next 372
2021-08-09 03:52:38.105544: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af11a6000 of size 425984 by op Tile_93 action_count 94195468133248 step 0 next 1447
2021-08-09 03:52:38.105573: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af120e000 of size 425984 by op Tile_18 action_count 94195468133249 step 0 next 142
2021-08-09 03:52:38.105601: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af1276000 of size 425984 by op Tile_36 action_count 94195468133250 step 0 next 48
2021-08-09 03:52:38.105627: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af12de000 of size 425984 by op Tile_54 action_count 94195468133251 step 0 next 554
2021-08-09 03:52:38.105661: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af1346000 of size 425984 by op Tile_72 action_count 94195468133252 step 0 next 2450
2021-08-09 03:52:38.105688: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af13ae000 of size 167168 by op Fill action_count 94195480111161 step 0 next 1695
2021-08-09 03:52:38.105715: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af13d6d00 of size 167168 by op Fill action_count 94195480111251 step 0 next 801
2021-08-09 03:52:38.105741: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af13ffa00 of size 167168 by op Add action_count 94195480110833 step 0 next 1896
2021-08-09 03:52:38.105779: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af1428700 of size 334336 by op Fill action_count 94195480111165 step 0 next 2100
2021-08-09 03:52:38.105806: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af147a100 of size 334336 by op Add action_count 94195480110847 step 0 next 243
2021-08-09 03:52:38.105832: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af14cbb00 of size 668416 by op Fill action_count 94195480111169 step 0 next 1256
2021-08-09 03:52:38.105854: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af156ee00 of size 668416 by op Add action_count 94195480111087 step 0 next 885
2021-08-09 03:52:38.105891: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af1612100 of size 668416 by op Add action_count 94195480110861 step 0 next 69
2021-08-09 03:52:38.105917: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af16b5400 of size 1336832 by op Add action_count 94195480111075 step 0 next 522
2021-08-09 03:52:38.105943: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af17fba00 of size 1336832 by op Add action_count 94195480110875 step 0 next 2605
2021-08-09 03:52:38.105970: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af1942000 of size 2673408 by op Add action_count 94195480111049 step 0 next 1528
2021-08-09 03:52:38.106007: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af1bceb00 of size 2673408 by op Fill action_count 94195480111173 step 0 next 420
2021-08-09 03:52:38.106037: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af1e5b600 of size 2673408 by op Add action_count 94195480110887 step 0 next 1731
2021-08-09 03:52:38.106064: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af20e8100 of size 2899968 by op Fill action_count 94195480111177 step 0 next 2014
2021-08-09 03:52:38.106090: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af23ac100 of size 2899968 by op Add action_count 94195480110901 step 0 next 997
2021-08-09 03:52:38.106127: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af2670100 of size 3145728 by op Add action_count 94195480111023 step 0 next 2104
2021-08-09 03:52:38.106156: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af2970100 of size 3145728 by op Add action_count 94195480110913 step 0 next 90
2021-08-09 03:52:38.106184: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af2c70100 of size 2097152 by op Add action_count 94195480110985 step 0 next 346
2021-08-09 03:52:38.106211: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af2e70100 of size 891136 by op Add action_count 94195480111061 step 0 next 51
2021-08-09 03:52:38.106248: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af2f49a00 of size 1206016 by op Fill action_count 94195480111233 step 0 next 651
2021-08-09 03:52:38.106275: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af3070100 of size 4194304 by op Add action_count 94195480110949 step 0 next 1880
2021-08-09 03:52:38.106301: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af3470100 of size 4194304 by op Add action_count 94195480110973 step 0 next 125
2021-08-09 03:52:38.106338: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af3870100 of size 6291456 by op Add action_count 94195480110925 step 0 next 787
2021-08-09 03:52:38.106367: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af3e70100 of size 3145728 by op Add action_count 94195480111011 step 0 next 1250
2021-08-09 03:52:38.106396: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af4170100 of size 4010240 by op Fill action_count 94195480111181 step 0 next 282
2021-08-09 03:52:38.106433: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af4543200 of size 5426944 by op Add action_count 94195480111037 step 0 next 1109
2021-08-09 03:52:38.106463: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af4a70100 of size 14162944 by op Add action_count 94195480110937 step 0 next 2163
2021-08-09 03:52:38.106489: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af57f1d00 of size 256 by op Sub action_count 94195129464775 step 0 next 135
2021-08-09 03:52:38.106531: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af57f1e00 of size 1099264 by op Add action_count 94195455013433 step 0 next 2920
2021-08-09 03:52:38.106562: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af58fe400 of size 2723840 by op Fill action_count 94195455013685 step 0 next 1741
2021-08-09 03:52:38.106588: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97400 of size 256 by op Sub action_count 94195347535728 step 0 next 312
2021-08-09 03:52:38.106624: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97500 of size 512 by op Reshape_92 action_count 94195455584276 step 0 next 1165
2021-08-09 03:52:38.106653: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97700 of size 256 by op Cast_181/x action_count 94195455584282 step 0 next 1894
2021-08-09 03:52:38.106685: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97800 of size 256 by op unet_depth3/encode0/batch_normalization_546/batchnorm/add/y action_count 94195455584283 step 0 next 2269
2021-08-09 03:52:38.106722: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97900 of size 256 by op Cast_180/x action_count 94195455584284 step 0 next 2460
2021-08-09 03:52:38.106751: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97a00 of size 256 by op assert_greater_equal/Assert/AssertGuard/pivot_f/_3 action_count 94195455584285 step 0 next 669
2021-08-09 03:52:38.106779: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97b00 of size 256 by op assert_greater_equal/Assert/AssertGuard/pivot_t/_4 action_count 94195455584286 step 0 next 2847
2021-08-09 03:52:38.106814: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97c00 of size 256 by op Const_35 action_count 94195455584287 step 0 next 2542
2021-08-09 03:52:38.106843: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97d00 of size 256 by op assert_less_equal/Assert/AssertGuard/pivot_f/_13 action_count 94195455584288 step 0 next 2314
2021-08-09 03:52:38.106870: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97e00 of size 256 by op assert_less_equal/Assert/AssertGuard/pivot_t/_14 action_count 94195455584289 step 0 next 2364
2021-08-09 03:52:38.106897: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b97f00 of size 256 by op assert_greater_equal_1/Assert/AssertGuard/pivot_f/_31 action_count 94195455584290 step 0 next 2687
2021-08-09 03:52:38.106933: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98000 of size 256 by op assert_greater_equal_1/Assert/AssertGuard/pivot_t/_32 action_count 94195455584291 step 0 next 2565
2021-08-09 03:52:38.106963: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98100 of size 256 by op assert_less_equal_1/Assert/AssertGuard/pivot_f/_41 action_count 94195455584292 step 0 next 2320
2021-08-09 03:52:38.106990: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98200 of size 256 by op assert_less_equal_1/Assert/AssertGuard/pivot_t/_42 action_count 94195455584293 step 0 next 1703
2021-08-09 03:52:38.107017: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98300 of size 256 by op assert_greater_equal_2/Assert/AssertGuard/pivot_f/_59 action_count 94195455584294 step 0 next 2452
2021-08-09 03:52:38.107053: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98400 of size 256 by op assert_greater_equal_2/Assert/AssertGuard/pivot_t/_60 action_count 94195455584295 step 0 next 3058
2021-08-09 03:52:38.107081: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98500 of size 256 by op AssignVariableOp action_count 94195467505291 step 0 next 2676
2021-08-09 03:52:38.107107: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98600 of size 256 by op AssignVariableOp action_count 94195480053202 step 0 next 2558
2021-08-09 03:52:38.107129: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98700 of size 256 by op AssignVariableOp action_count 94195467505325 step 0 next 779
2021-08-09 03:52:38.107162: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98800 of size 256 by op AssignVariableOp action_count 94195467505327 step 0 next 2591
2021-08-09 03:52:38.107191: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98900 of size 512 by op AssignVariableOp action_count 94195467505337 step 0 next 797
2021-08-09 03:52:38.107217: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98b00 of size 768 by op Fill action_count 94195467562906 step 0 next 716
2021-08-09 03:52:38.107243: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98e00 of size 256 by op Sub action_count 94195347535757 step 0 next 485
2021-08-09 03:52:38.107265: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b98f00 of size 256 by op Sub action_count 94195347535770 step 0 next 233
2021-08-09 03:52:38.107301: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b99000 of size 256 by op Sub action_count 94195347535771 step 0 next 1231
2021-08-09 03:52:38.107328: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af5b99100 of size 6291456 by op Add action_count 94195480110961 step 0 next 1088
2021-08-09 03:52:38.107355: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af6199100 of size 6045696 by op Add action_count 94195480110999 step 0 next 445
2021-08-09 03:52:38.107382: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af675d100 of size 3145728 by op Fill action_count 94195480111185 step 0 next 460
2021-08-09 03:52:38.107408: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af6a5d100 of size 6291456 by op Fill action_count 94195480111189 step 0 next 78
2021-08-09 03:52:38.107445: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af705d100 of size 12582912 by op Fill action_count 94195480111193 step 0 next 1485
2021-08-09 03:52:38.107473: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af7c5d100 of size 4194304 by op Fill action_count 94195480111197 step 0 next 694
2021-08-09 03:52:38.107499: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af805d100 of size 6291456 by op Fill action_count 94195480111201 step 0 next 1071
2021-08-09 03:52:38.107528: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af865d100 of size 3145728 by op Fill action_count 94195480111205 step 0 next 1285
2021-08-09 03:52:38.107563: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af895d100 of size 2097152 by op Fill action_count 94195480111209 step 0 next 627
2021-08-09 03:52:38.107592: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af8b5d100 of size 6045696 by op Fill action_count 94195480111213 step 0 next 1817
2021-08-09 03:52:38.107620: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af9121100 of size 3145728 by op Fill action_count 94195480111217 step 0 next 531
2021-08-09 03:52:38.107648: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af9421100 of size 1933312 by op Fill action_count 94195480111221 step 0 next 1690
2021-08-09 03:52:38.107685: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af95f9100 of size 4010240 by op Fill action_count 94195480111225 step 0 next 34
2021-08-09 03:52:38.107714: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af99cc200 of size 2673408 by op Fill action_count 94195480111229 step 0 next 2339
2021-08-09 03:52:38.107744: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af9c58d00 of size 1002752 by op Fill action_count 94195480111237 step 0 next 202
2021-08-09 03:52:38.107771: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af9d4da00 of size 668416 by op Fill action_count 94195480111241 step 0 next 1126
2021-08-09 03:52:38.107800: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af9df0d00 of size 334336 by op Fill action_count 94195480111255 step 0 next 753
2021-08-09 03:52:38.107827: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af9e42700 of size 668416 by op Fill action_count 94195480111259 step 0 next 99
2021-08-09 03:52:38.107854: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3af9ee5a00 of size 1336832 by op Fill action_count 94195480111263 step 0 next 70
2021-08-09 03:52:38.107880: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afa02c000 of size 2673408 by op Fill action_count 94195480111267 step 0 next 483
2021-08-09 03:52:38.107904: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afa2b8b00 of size 2899968 by op Fill action_count 94195480111271 step 0 next 1603
2021-08-09 03:52:38.107926: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afa57cb00 of size 3145728 by op Fill action_count 94195480111275 step 0 next 1661
2021-08-09 03:52:38.107950: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afa87cb00 of size 6291456 by op Fill action_count 94195480111279 step 0 next 1123
2021-08-09 03:52:38.107988: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afae7cb00 of size 12582912 by op Fill action_count 94195480111283 step 0 next 92
2021-08-09 03:52:38.108018: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afba7cb00 of size 4194304 by op Fill action_count 94195480111287 step 0 next 1326
2021-08-09 03:52:38.108046: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afbe7cb00 of size 6291456 by op Fill action_count 94195480111291 step 0 next 695
2021-08-09 03:52:38.108073: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afc47cb00 of size 3145728 by op Fill action_count 94195480111295 step 0 next 626
2021-08-09 03:52:38.108099: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afc77cb00 of size 2097152 by op Fill action_count 94195480111299 step 0 next 1067
2021-08-09 03:52:38.108126: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afc97cb00 of size 6045696 by op Fill action_count 94195480111303 step 0 next 1103
2021-08-09 03:52:38.108162: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afcf40b00 of size 2048 by op Fill action_count 94195480111305 step 0 next 472
2021-08-09 03:52:38.108190: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afcf41300 of size 2048 by op Fill action_count 94195480111306 step 0 next 1367
2021-08-09 03:52:38.108216: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afcf41b00 of size 3145728 by op Fill action_count 94195480111307 step 0 next 750
2021-08-09 03:52:38.108238: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd241b00 of size 2048 by op Fill action_count 94195480111308 step 0 next 286
2021-08-09 03:52:38.108262: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd242300 of size 2048 by op Fill action_count 94195480111309 step 0 next 658
2021-08-09 03:52:38.108284: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd242b00 of size 2048 by op Fill action_count 94195480111310 step 0 next 1090
2021-08-09 03:52:38.108316: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd243300 of size 1933312 by op Fill action_count 94195480111311 step 0 next 370
2021-08-09 03:52:38.108343: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd41b300 of size 2048 by op Fill action_count 94195480111312 step 0 next 1555
2021-08-09 03:52:38.108369: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd41bb00 of size 2048 by op Fill action_count 94195480111313 step 0 next 947
2021-08-09 03:52:38.108394: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd41c300 of size 2048 by op Fill action_count 94195480111314 step 0 next 1504
2021-08-09 03:52:38.108421: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd41cb00 of size 4010240 by op Fill action_count 94195480111315 step 0 next 1336
2021-08-09 03:52:38.108450: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd7efc00 of size 2048 by op Fill action_count 94195480111316 step 0 next 492
2021-08-09 03:52:38.108476: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd7f0400 of size 2048 by op Fill action_count 94195480111317 step 0 next 1171
2021-08-09 03:52:38.108503: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd7f0c00 of size 2048 by op Fill action_count 94195480111318 step 0 next 469
2021-08-09 03:52:38.108541: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afd7f1400 of size 2673408 by op Fill action_count 94195480111319 step 0 next 1529
2021-08-09 03:52:38.108570: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afda7df00 of size 2048 by op Fill action_count 94195480111320 step 0 next 948
2021-08-09 03:52:38.108607: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afda7e700 of size 2048 by op Fill action_count 94195480111321 step 0 next 1599
2021-08-09 03:52:38.108636: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afda7ef00 of size 2048 by op Fill action_count 94195480111322 step 0 next 1009
2021-08-09 03:52:38.108662: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afda7f700 of size 891136 by op Fill action_count 94195480111323 step 0 next 569
2021-08-09 03:52:38.108689: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdb59000 of size 1024 by op Fill action_count 94195480111324 step 0 next 1520
2021-08-09 03:52:38.108717: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdb59400 of size 1024 by op Fill action_count 94195480111325 step 0 next 463
2021-08-09 03:52:38.108745: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdb59800 of size 1024 by op Fill action_count 94195480111326 step 0 next 1293
2021-08-09 03:52:38.108780: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdb59c00 of size 1002752 by op Fill action_count 94195480111327 step 0 next 1486
2021-08-09 03:52:38.108808: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdc4e900 of size 1024 by op Fill action_count 94195480111328 step 0 next 1642
2021-08-09 03:52:38.108830: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdc4ed00 of size 1024 by op Fill action_count 94195480111329 step 0 next 296
2021-08-09 03:52:38.108854: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdc4f100 of size 1024 by op Fill action_count 94195480111330 step 0 next 654
2021-08-09 03:52:38.108882: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdc4f500 of size 668416 by op Fill action_count 94195480111331 step 0 next 187
2021-08-09 03:52:38.108908: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf2800 of size 1024 by op Fill action_count 94195480111332 step 0 next 1851
2021-08-09 03:52:38.108934: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf2c00 of size 1024 by op Fill action_count 94195480111333 step 0 next 713
2021-08-09 03:52:38.108962: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3000 of size 1024 by op Fill action_count 94195480111334 step 0 next 116
2021-08-09 03:52:38.108995: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3400 of size 1024 by op Fill action_count 94195480111335 step 0 next 2017
2021-08-09 03:52:38.109022: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3800 of size 256 by op Fill action_count 94195480111336 step 0 next 1222
2021-08-09 03:52:38.109058: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3900 of size 256 by op ConstantFolding/truediv_recip action_count 94195480111337 step 0 next 1538
2021-08-09 03:52:38.109088: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3a00 of size 256 by op ConstantFolding/gradient_tape/unet_depth4/encode0/batch_normalization_580/moments/truediv_recip action_count 94
195480111338 step 0 next 563
2021-08-09 03:52:38.109118: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3b00 of size 256 by op ConstantFolding/gradient_tape/unet_depth4/encode1/batch_normalization_582/moments/truediv_recip action_count 94
195480111339 step 0 next 1699
2021-08-09 03:52:38.109148: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3c00 of size 256 by op ConstantFolding/gradient_tape/unet_depth4/encode2/batch_normalization_584/moments/truediv_recip action_count 94
195480111340 step 0 next 960
2021-08-09 03:52:38.109178: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3d00 of size 256 by op ConstantFolding/gradient_tape/unet_depth4/encode3/batch_normalization_586/moments/truediv_recip action_count 94
195480111341 step 0 next 1704
2021-08-09 03:52:38.109207: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3e00 of size 256 by op ConstantFolding/gradient_tape/unet_depth4/two_conv_center/batch_normalization_588/moments/truediv_recip action_
count 94195480111342 step 0 next 1345
2021-08-09 03:52:38.109234: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf3f00 of size 256 by op Adam/add/y action_count 94195480111343 step 0 next 1887
2021-08-09 03:52:38.109261: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf4000 of size 256 by op add_16/y action_count 94195480111344 step 0 next 1579
2021-08-09 03:52:38.109298: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf4100 of size 256 by op gradient_tape/binary_ce_dice/weighted_loss/Reshape_1 action_count 94195480111345 step 0 next 2117
2021-08-09 03:52:38.109329: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf4200 of size 256 by op gradient_tape/binary_ce_dice/logistic_loss/zeros_like/Const action_count 94195480111346 step 0 next 690
2021-08-09 03:52:38.109355: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf4300 of size 256 by op Sum_1 action_count 94195480111347 step 0 next 1701
2021-08-09 03:52:38.109391: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdcf4400 of size 1441792 by op gradient_tape/binary_ce_dice/logistic_loss/zeros_like action_count 94195480111348 step 0 next 2114
2021-08-09 03:52:38.109420: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afde54400 of size 256 by op Adam/gradients/ones action_count 94195480111349 step 0 next 1058
2021-08-09 03:52:38.109447: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afde54500 of size 256 by op truediv/y action_count 94195480111350 step 0 next 1822
2021-08-09 03:52:38.109473: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afde54600 of size 256 by op binary_ce_dice/add_1/y action_count 94195480111351 step 0 next 17
2021-08-09 03:52:38.109510: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afde54700 of size 1441792 by op Tile_93 action_count 94195480111352 step 0 next 223
2021-08-09 03:52:38.109544: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdfb4700 of size 256 by op binary_ce_dice/weighted_loss/num_elements/Cast action_count 94195480111353 step 0 next 2135
2021-08-09 03:52:38.109571: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afdfb4800 of size 1441792 by op Tile_18 action_count 94195480111354 step 0 next 692
2021-08-09 03:52:38.109597: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe114800 of size 1441792 by op Tile_36 action_count 94195480111355 step 0 next 1499
2021-08-09 03:52:38.109619: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe274800 of size 1441792 by op Tile_54 action_count 94195480111356 step 0 next 1747
2021-08-09 03:52:38.109644: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe3d4800 of size 1441792 by op Tile_72 action_count 94195480111357 step 0 next 1220
2021-08-09 03:52:38.109672: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe534800 of size 1441792 by op Tile action_count 94195480111358 step 0 next 1840
2021-08-09 03:52:38.109709: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe694800 of size 512 by op Reshape_92 action_count 94195480111359 step 0 next 941
2021-08-09 03:52:38.109740: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe694a00 of size 256 by op unet_depth4/conv_transpose_decoder0/batch_normalization_599/AssignMovingAvg/decay action_count 94195480111360 s
tep 0 next 283
2021-08-09 03:52:38.109767: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe694b00 of size 256 by op unet_depth4/two_conv_decoder0/batch_normalization_601/batchnorm/add/y action_count 94195480111361 step 0 next 3
86
2021-08-09 03:52:38.109794: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe694c00 of size 256 by op gradient_tape/binary_ce_dice/logistic_loss/add/x action_count 94195480111362 step 0 next 1243
2021-08-09 03:52:38.109820: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe694d00 of size 256 by op assert_greater_equal/Assert/AssertGuard/pivot_f/_3 action_count 94195480111363 step 0 next 2145
2021-08-09 03:52:38.109847: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe694e00 of size 256 by op assert_greater_equal/Assert/AssertGuard/pivot_t/_4 action_count 94195480111364 step 0 next 1275
2021-08-09 03:52:38.109884: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe694f00 of size 256 by op assert_less_equal/Assert/AssertGuard/pivot_f/_13 action_count 94195480111365 step 0 next 1304
2021-08-09 03:52:38.109912: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695000 of size 256 by op assert_less_equal/Assert/AssertGuard/pivot_t/_14 action_count 94195480111366 step 0 next 1317
2021-08-09 03:52:38.109938: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695100 of size 256 by op Const_35 action_count 94195480111367 step 0 next 720
2021-08-09 03:52:38.109965: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695200 of size 256 by op ArithmeticOptimizer/AddOpsRewrite_add_16/tmp_var_zeros action_count 94195480111368 step 0 next 1796
2021-08-09 03:52:38.109991: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695300 of size 256 by op assert_greater_equal_1/Assert/AssertGuard/pivot_f/_31 action_count 94195480111369 step 0 next 1288
2021-08-09 03:52:38.110018: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695400 of size 256 by op assert_greater_equal_1/Assert/AssertGuard/pivot_t/_32 action_count 94195480111370 step 0 next 1833
2021-08-09 03:52:38.110052: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695500 of size 256 by op assert_less_equal_1/Assert/AssertGuard/pivot_f/_41 action_count 94195480111371 step 0 next 1133
2021-08-09 03:52:38.110080: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695600 of size 256 by op assert_less_equal_1/Assert/AssertGuard/pivot_t/_42 action_count 94195480111372 step 0 next 474
2021-08-09 03:52:38.110109: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695700 of size 256 by op gradient_tape/unet_depth4/two_conv_decoder0/batch_normalization_601/moments/scalar action_count 94195480111373
step 0 next 224
2021-08-09 03:52:38.110137: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695800 of size 256 by op assert_greater_equal_2/Assert/AssertGuard/pivot_f/_59 action_count 94195480111375 step 0 next 1522
2021-08-09 03:52:38.110163: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695900 of size 256 by op assert_greater_equal_2/Assert/AssertGuard/pivot_t/_60 action_count 94195480111376 step 0 next 598
2021-08-09 03:52:38.110190: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695a00 of size 256 by op assert_less_equal_2/Assert/AssertGuard/pivot_f/_69 action_count 94195480111377 step 0 next 1673
2021-08-09 03:52:38.110226: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695b00 of size 256 by op assert_less_equal_2/Assert/AssertGuard/pivot_t/_70 action_count 94195480111378 step 0 next 1565
2021-08-09 03:52:38.110257: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695c00 of size 256 by op assert_greater_equal_3/Assert/AssertGuard/pivot_f/_87 action_count 94195480111379 step 0 next 1719
2021-08-09 03:52:38.110282: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695d00 of size 256 by op assert_greater_equal_3/Assert/AssertGuard/pivot_t/_88 action_count 94195480111380 step 0 next 1730
2021-08-09 03:52:38.110309: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695e00 of size 256 by op assert_less_equal_3/Assert/AssertGuard/pivot_f/_97 action_count 94195480111381 step 0 next 1514
2021-08-09 03:52:38.110337: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe695f00 of size 256 by op assert_less_equal_3/Assert/AssertGuard/pivot_t/_98 action_count 94195480111382 step 0 next 1671
2021-08-09 03:52:38.110365: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696000 of size 256 by op gradient_tape/unet_depth4/two_conv_decoder0/batch_normalization_600/moments/scalar action_count 94195480111383
step 0 next 1576
2021-08-09 03:52:38.110403: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696100 of size 256 by op assert_greater_equal_4/Assert/AssertGuard/pivot_f/_115 action_count 94195480111384 step 0 next 1597
2021-08-09 03:52:38.110431: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696200 of size 256 by op assert_greater_equal_4/Assert/AssertGuard/pivot_t/_116 action_count 94195480111385 step 0 next 1512
2021-08-09 03:52:38.110458: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696300 of size 256 by op assert_less_equal_4/Assert/AssertGuard/pivot_f/_125 action_count 94195480111386 step 0 next 1558
2021-08-09 03:52:38.110484: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696400 of size 256 by op assert_less_equal_4/Assert/AssertGuard/pivot_t/_126 action_count 94195480111387 step 0 next 1539
2021-08-09 03:52:38.110511: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696500 of size 256 by op assert_greater_equal_5/Assert/AssertGuard/pivot_f/_143 action_count 94195480111388 step 0 next 1536
2021-08-09 03:52:38.110545: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696600 of size 256 by op assert_greater_equal_5/Assert/AssertGuard/pivot_t/_144 action_count 94195480111389 step 0 next 1590
2021-08-09 03:52:38.110583: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696700 of size 256 by op gradient_tape/unet_depth4/conv_transpose_decoder0/batch_normalization_599/moments/scalar action_count 941954801
11390 step 0 next 1725
2021-08-09 03:52:38.110613: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696800 of size 256 by op assert_less_equal_5/Assert/AssertGuard/pivot_f/_153 action_count 94195480111391 step 0 next 1676
2021-08-09 03:52:38.110642: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696900 of size 256 by op assert_less_equal_5/Assert/AssertGuard/pivot_t/_154 action_count 94195480111392 step 0 next 1632
2021-08-09 03:52:38.110669: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696a00 of size 256 by op assert_greater_equal_6/Assert/AssertGuard/pivot_f/_171 action_count 94195480111393 step 0 next 1532
2021-08-09 03:52:38.110696: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696b00 of size 256 by op assert_greater_equal_6/Assert/AssertGuard/pivot_t/_172 action_count 94195480111394 step 0 next 1718
2021-08-09 03:52:38.110722: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696c00 of size 256 by op assert_less_equal_6/Assert/AssertGuard/pivot_f/_181 action_count 94195480111395 step 0 next 427
2021-08-09 03:52:38.110758: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696d00 of size 256 by op assert_less_equal_6/Assert/AssertGuard/pivot_t/_182 action_count 94195480111396 step 0 next 1553
2021-08-09 03:52:38.110788: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696e00 of size 256 by op gradient_tape/unet_depth4/two_conv_decoder1/batch_normalization_598/moments/scalar action_count 94195480111397
step 0 next 516
2021-08-09 03:52:38.110814: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe696f00 of size 256 by op assert_greater_equal_7/Assert/AssertGuard/pivot_f/_199 action_count 94195480111398 step 0 next 1550
2021-08-09 03:52:38.110841: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697000 of size 256 by op assert_greater_equal_7/Assert/AssertGuard/pivot_t/_200 action_count 94195480111399 step 0 next 129
2021-08-09 03:52:38.110867: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697100 of size 256 by op assert_less_equal_7/Assert/AssertGuard/pivot_f/_209 action_count 94195480111400 step 0 next 369
2021-08-09 03:52:38.110894: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697200 of size 256 by op assert_less_equal_7/Assert/AssertGuard/pivot_t/_210 action_count 94195480111401 step 0 next 1542
2021-08-09 03:52:38.110920: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697300 of size 256 by op assert_greater_equal_8/Assert/AssertGuard/pivot_f/_227 action_count 94195480111402 step 0 next 433
2021-08-09 03:52:38.110958: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697400 of size 256 by op assert_greater_equal_8/Assert/AssertGuard/pivot_t/_228 action_count 94195480111403 step 0 next 1678
2021-08-09 03:52:38.110986: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697500 of size 256 by op gradient_tape/unet_depth4/two_conv_decoder1/batch_normalization_597/moments/scalar action_count 94195480111404
step 0 next 241
2021-08-09 03:52:38.111015: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697600 of size 256 by op assert_less_equal_8/Assert/AssertGuard/pivot_f/_237 action_count 94195480111405 step 0 next 1776
2021-08-09 03:52:38.111052: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697700 of size 256 by op assert_less_equal_8/Assert/AssertGuard/pivot_t/_238 action_count 94195480111406 step 0 next 1568
2021-08-09 03:52:38.111081: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697800 of size 256 by op assert_greater_equal_9/Assert/AssertGuard/pivot_f/_255 action_count 94195480111407 step 0 next 951
2021-08-09 03:52:38.111109: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697900 of size 256 by op assert_greater_equal_9/Assert/AssertGuard/pivot_t/_256 action_count 94195480111408 step 0 next 1720
2021-08-09 03:52:38.111135: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697a00 of size 256 by op assert_less_equal_9/Assert/AssertGuard/pivot_f/_265 action_count 94195480111409 step 0 next 538
2021-08-09 03:52:38.111161: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697b00 of size 256 by op assert_less_equal_9/Assert/AssertGuard/pivot_t/_266 action_count 94195480111410 step 0 next 1287
2021-08-09 03:52:38.111190: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697c00 of size 256 by op assert_greater_equal_10/Assert/AssertGuard/pivot_f/_283 action_count 94195480111411 step 0 next 549
2021-08-09 03:52:38.111218: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697d00 of size 256 by op assert_greater_equal_10/Assert/AssertGuard/pivot_t/_284 action_count 94195480111412 step 0 next 605
2021-08-09 03:52:38.111243: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697e00 of size 256 by op gradient_tape/unet_depth4/conv_transpose_decoder1/batch_normalization_596/moments/scalar action_count 941954801
11413 step 0 next 1567
2021-08-09 03:52:38.111283: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe697f00 of size 256 by op assert_less_equal_10/Assert/AssertGuard/pivot_f/_293 action_count 94195480111414 step 0 next 111
2021-08-09 03:52:38.111311: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698000 of size 256 by op assert_less_equal_10/Assert/AssertGuard/pivot_t/_294 action_count 94195480111415 step 0 next 390
2021-08-09 03:52:38.111340: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698100 of size 256 by op assert_greater_equal_11/Assert/AssertGuard/pivot_f/_311 action_count 94195480111416 step 0 next 907
2021-08-09 03:52:38.111366: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698200 of size 256 by op assert_greater_equal_11/Assert/AssertGuard/pivot_t/_312 action_count 94195480111417 step 0 next 1209
2021-08-09 03:52:38.111393: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698300 of size 256 by op assert_less_equal_11/Assert/AssertGuard/pivot_f/_321 action_count 94195480111418 step 0 next 289
2021-08-09 03:52:38.111430: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698400 of size 256 by op assert_less_equal_11/Assert/AssertGuard/pivot_t/_322 action_count 94195480111419 step 0 next 1891
2021-08-09 03:52:38.111458: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698500 of size 256 by op gradient_tape/unet_depth4/two_conv_decoder2/batch_normalization_595/moments/scalar action_count 94195480111420
step 0 next 268
2021-08-09 03:52:38.111485: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698600 of size 256 by op assert_greater_equal_12/Assert/AssertGuard/pivot_f/_339 action_count 94195480111421 step 0 next 118
2021-08-09 03:52:38.111511: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698700 of size 256 by op assert_greater_equal_12/Assert/AssertGuard/pivot_t/_340 action_count 94195480111422 step 0 next 1145
2021-08-09 03:52:38.111545: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698800 of size 256 by op assert_less_equal_12/Assert/AssertGuard/pivot_f/_349 action_count 94195480111423 step 0 next 893
2021-08-09 03:52:38.111574: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698900 of size 256 by op assert_less_equal_12/Assert/AssertGuard/pivot_t/_350 action_count 94195480111424 step 0 next 1114
2021-08-09 03:52:38.111603: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698a00 of size 256 by op assert_greater_equal_13/Assert/AssertGuard/pivot_f/_367 action_count 94195480111425 step 0 next 1753
2021-08-09 03:52:38.111640: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698b00 of size 256 by op assert_greater_equal_13/Assert/AssertGuard/pivot_t/_368 action_count 94195480111426 step 0 next 1264
2021-08-09 03:52:38.111667: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698c00 of size 256 by op gradient_tape/unet_depth4/two_conv_decoder2/batch_normalization_594/moments/scalar action_count 94195480111427
step 0 next 475
2021-08-09 03:52:38.111706: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698d00 of size 256 by op assert_less_equal_13/Assert/AssertGuard/pivot_f/_377 action_count 94195480111428 step 0 next 1060
2021-08-09 03:52:38.111730: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698e00 of size 256 by op assert_less_equal_13/Assert/AssertGuard/pivot_t/_378 action_count 94195480111429 step 0 next 1700
2021-08-09 03:52:38.111755: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe698f00 of size 256 by op assert_greater_equal_14/Assert/AssertGuard/pivot_f/_395 action_count 94195480111430 step 0 next 1755
2021-08-09 03:52:38.111789: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699000 of size 256 by op assert_greater_equal_14/Assert/AssertGuard/pivot_t/_396 action_count 94195480111431 step 0 next 2062
2021-08-09 03:52:38.111813: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699100 of size 256 by op assert_less_equal_14/Assert/AssertGuard/pivot_f/_405 action_count 94195480111432 step 0 next 1877
2021-08-09 03:52:38.111838: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699200 of size 256 by op assert_less_equal_14/Assert/AssertGuard/pivot_t/_406 action_count 94195480111433 step 0 next 1596
2021-08-09 03:52:38.111862: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699300 of size 256 by op gradient_tape/unet_depth4/conv_transpose_decoder2/batch_normalization_593/moments/scalar action_count 941954801
11434 step 0 next 1140
2021-08-09 03:52:38.111888: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699400 of size 256 by op assert_greater_equal_15/Assert/AssertGuard/pivot_f/_423 action_count 94195480111435 step 0 next 418
2021-08-09 03:52:38.111913: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699500 of size 256 by op assert_greater_equal_15/Assert/AssertGuard/pivot_t/_424 action_count 94195480111436 step 0 next 2194
2021-08-09 03:52:38.111947: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699600 of size 256 by op assert_less_equal_15/Assert/AssertGuard/pivot_f/_433 action_count 94195480111437 step 0 next 876
2021-08-09 03:52:38.111975: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699700 of size 256 by op assert_less_equal_15/Assert/AssertGuard/pivot_t/_434 action_count 94195480111438 step 0 next 387
2021-08-09 03:52:38.112003: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699800 of size 256 by op assert_greater_equal_16/Assert/AssertGuard/pivot_f/_451 action_count 94195480111439 step 0 next 2048
2021-08-09 03:52:38.112030: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699900 of size 256 by op assert_greater_equal_16/Assert/AssertGuard/pivot_t/_452 action_count 94195480111440 step 0 next 1297
2021-08-09 03:52:38.112060: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699a00 of size 256 by op gradient_tape/unet_depth4/two_conv_decoder3/batch_normalization_592/moments/scalar action_count 94195480111441
step 0 next 1552
2021-08-09 03:52:38.112095: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699b00 of size 256 by op assert_less_equal_16/Assert/AssertGuard/pivot_f/_461 action_count 94195480111442 step 0 next 1858
2021-08-09 03:52:38.112124: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699c00 of size 256 by op assert_less_equal_16/Assert/AssertGuard/pivot_t/_462 action_count 94195480111443 step 0 next 1274
2021-08-09 03:52:38.112150: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699d00 of size 256 by op assert_greater_equal_17/Assert/AssertGuard/pivot_f/_479 action_count 94195480111444 step 0 next 2097
2021-08-09 03:52:38.112177: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699e00 of size 256 by op assert_greater_equal_17/Assert/AssertGuard/pivot_t/_480 action_count 94195480111445 step 0 next 55
2021-08-09 03:52:38.112203: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe699f00 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_f/_489 action_count 94195480111446 step 0 next 1333
2021-08-09 03:52:38.112230: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a000 of size 256 by op assert_less_equal_17/Assert/AssertGuard/pivot_t/_490 action_count 94195480111447 step 0 next 1263
2021-08-09 03:52:38.112256: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a100 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_f/_507 action_count 94195480111448 step 0 next 1277
2021-08-09 03:52:38.112286: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a200 of size 256 by op assert_greater_equal_18/Assert/AssertGuard/pivot_t/_508 action_count 94195480111449 step 0 next 1079
2021-08-09 03:52:38.112314: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a300 of size 256 by op gradient_tape/unet_depth4/two_conv_decoder3/batch_normalization_591/moments/scalar action_count 94195480111450
step 0 next 1543
2021-08-09 03:52:38.112339: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a400 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_f/_517 action_count 94195480111451 step 0 next 1722
2021-08-09 03:52:38.112365: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a500 of size 256 by op assert_less_equal_18/Assert/AssertGuard/pivot_t/_518 action_count 94195480111452 step 0 next 2614
2021-08-09 03:52:38.112403: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a600 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_f/_535 action_count 94195480111453 step 0 next 365
2021-08-09 03:52:38.112429: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a700 of size 256 by op assert_greater_equal_19/Assert/AssertGuard/pivot_t/_536 action_count 94195480111454 step 0 next 1068
2021-08-09 03:52:38.112456: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a800 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_f/_545 action_count 94195480111455 step 0 next 900
2021-08-09 03:52:38.112481: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69a900 of size 256 by op assert_less_equal_19/Assert/AssertGuard/pivot_t/_546 action_count 94195480111456 step 0 next 802
2021-08-09 03:52:38.112508: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69aa00 of size 256 by op gradient_tape/unet_depth4/conv_transpose_decoder3/batch_normalization_590/moments/scalar action_count 941954801
11457 step 0 next 2922
2021-08-09 03:52:38.112542: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69ab00 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_f/_563 action_count 94195480111458 step 0 next 2994
2021-08-09 03:52:38.112566: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69ac00 of size 256 by op assert_greater_equal_20/Assert/AssertGuard/pivot_t/_564 action_count 94195480111459 step 0 next 1443
2021-08-09 03:52:38.112589: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69ad00 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_f/_573 action_count 94195480111460 step 0 next 3039
2021-08-09 03:52:38.112612: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69ae00 of size 256 by op assert_less_equal_20/Assert/AssertGuard/pivot_t/_574 action_count 94195480111461 step 0 next 2814
2021-08-09 03:52:38.112646: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69af00 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_f/_591 action_count 94195480111462 step 0 next 2808
2021-08-09 03:52:38.112673: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b000 of size 256 by op assert_greater_equal_21/Assert/AssertGuard/pivot_t/_592 action_count 94195480111463 step 0 next 3093
2021-08-09 03:52:38.112697: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b100 of size 256 by op gradient_tape/unet_depth4/two_conv_center/batch_normalization_589/moments/scalar action_count 94195480111464 st
ep 0 next 2882
2021-08-09 03:52:38.112721: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b200 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_f/_601 action_count 94195480111465 step 0 next 2963
2021-08-09 03:52:38.112755: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b300 of size 256 by op assert_less_equal_21/Assert/AssertGuard/pivot_t/_602 action_count 94195480111466 step 0 next 1412
2021-08-09 03:52:38.112781: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b400 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_f/_619 action_count 94195480111467 step 0 next 2849
2021-08-09 03:52:38.112805: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b500 of size 256 by op assert_greater_equal_22/Assert/AssertGuard/pivot_t/_620 action_count 94195480111468 step 0 next 2795
2021-08-09 03:52:38.112830: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b600 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_f/_629 action_count 94195480111469 step 0 next 2932
2021-08-09 03:52:38.112856: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b700 of size 256 by op assert_less_equal_22/Assert/AssertGuard/pivot_t/_630 action_count 94195480111470 step 0 next 2770
2021-08-09 03:52:38.112880: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b800 of size 256 by op gradient_tape/unet_depth4/two_conv_center/batch_normalization_588/moments/scalar action_count 94195480111471 st
ep 0 next 2243
2021-08-09 03:52:38.112905: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69b900 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_f/_647 action_count 94195480111472 step 0 next 2759
2021-08-09 03:52:38.112929: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69ba00 of size 256 by op assert_greater_equal_23/Assert/AssertGuard/pivot_t/_648 action_count 94195480111473 step 0 next 2741
2021-08-09 03:52:38.112965: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69bb00 of size 256 by op assert_less_equal_23/Assert/AssertGuard/pivot_f/_657 action_count 94195480111474 step 0 next 443
2021-08-09 03:52:38.113001: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69bc00 of size 256 by op assert_less_equal_23/Assert/AssertGuard/pivot_t/_658 action_count 94195480111475 step 0 next 2897
2021-08-09 03:52:38.113027: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69bd00 of size 256 by op assert_greater_equal_24/Assert/AssertGuard/pivot_f/_675 action_count 94195480111476 step 0 next 2857
2021-08-09 03:52:38.113050: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69be00 of size 256 by op assert_greater_equal_24/Assert/AssertGuard/pivot_t/_676 action_count 94195480111477 step 0 next 3087
2021-08-09 03:52:38.113073: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69bf00 of size 256 by op assert_less_equal_24/Assert/AssertGuard/pivot_f/_685 action_count 94195480111478 step 0 next 3055
2021-08-09 03:52:38.113100: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c000 of size 256 by op assert_less_equal_24/Assert/AssertGuard/pivot_t/_686 action_count 94195480111479 step 0 next 2949
2021-08-09 03:52:38.113125: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c100 of size 256 by op gradient_tape/unet_depth4/encode3/batch_normalization_587/moments/scalar action_count 94195480111480 step 0 nex
t 2981
2021-08-09 03:52:38.113159: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c200 of size 256 by op assert_greater_equal_25/Assert/AssertGuard/pivot_f/_703 action_count 94195480111481 step 0 next 3083
2021-08-09 03:52:38.113180: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c300 of size 256 by op assert_greater_equal_25/Assert/AssertGuard/pivot_t/_704 action_count 94195480111482 step 0 next 3088
2021-08-09 03:52:38.113203: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c400 of size 256 by op assert_less_equal_25/Assert/AssertGuard/pivot_f/_713 action_count 94195480111483 step 0 next 2752
2021-08-09 03:52:38.113226: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c500 of size 256 by op assert_less_equal_25/Assert/AssertGuard/pivot_t/_714 action_count 94195480111484 step 0 next 3048
2021-08-09 03:52:38.113253: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c600 of size 256 by op assert_greater_equal_26/Assert/AssertGuard/pivot_f/_731 action_count 94195480111485 step 0 next 3002
2021-08-09 03:52:38.113285: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c700 of size 256 by op assert_greater_equal_26/Assert/AssertGuard/pivot_t/_732 action_count 94195480111486 step 0 next 349
2021-08-09 03:52:38.113310: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c800 of size 256 by op gradient_tape/unet_depth4/encode3/batch_normalization_586/moments/scalar action_count 94195480111487 step 0 nex
t 2859
2021-08-09 03:52:38.113334: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69c900 of size 256 by op assert_less_equal_26/Assert/AssertGuard/pivot_f/_741 action_count 94195480111488 step 0 next 2696
2021-08-09 03:52:38.113358: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69ca00 of size 256 by op assert_less_equal_26/Assert/AssertGuard/pivot_t/_742 action_count 94195480111489 step 0 next 2629
2021-08-09 03:52:38.113385: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69cb00 of size 256 by op assert_greater_equal_27/Assert/AssertGuard/pivot_f/_759 action_count 94195480111490 step 0 next 3008
2021-08-09 03:52:38.113405: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69cc00 of size 256 by op assert_greater_equal_27/Assert/AssertGuard/pivot_t/_760 action_count 94195480111491 step 0 next 2093
2021-08-09 03:52:38.113428: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69cd00 of size 256 by op assert_less_equal_27/Assert/AssertGuard/pivot_f/_769 action_count 94195480111492 step 0 next 2978
2021-08-09 03:52:38.113459: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69ce00 of size 256 by op assert_less_equal_27/Assert/AssertGuard/pivot_t/_770 action_count 94195480111493 step 0 next 3074
2021-08-09 03:52:38.113485: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69cf00 of size 256 by op assert_greater_equal_28/Assert/AssertGuard/pivot_f/_787 action_count 94195480111494 step 0 next 3075
2021-08-09 03:52:38.113509: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d000 of size 256 by op assert_greater_equal_28/Assert/AssertGuard/pivot_t/_788 action_count 94195480111495 step 0 next 3018
2021-08-09 03:52:38.113550: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d100 of size 256 by op gradient_tape/unet_depth4/encode2/batch_normalization_585/moments/scalar action_count 94195480111496 step 0 nex
t 2844
2021-08-09 03:52:38.113577: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d200 of size 256 by op assert_less_equal_28/Assert/AssertGuard/pivot_f/_797 action_count 94195480111497 step 0 next 1422
2021-08-09 03:52:38.113601: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d300 of size 256 by op assert_less_equal_28/Assert/AssertGuard/pivot_t/_798 action_count 94195480111498 step 0 next 3025
2021-08-09 03:52:38.113625: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d400 of size 256 by op assert_greater_equal_29/Assert/AssertGuard/pivot_f/_815 action_count 94195480111499 step 0 next 3003
2021-08-09 03:52:38.113649: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d500 of size 256 by op assert_greater_equal_29/Assert/AssertGuard/pivot_t/_816 action_count 94195480111500 step 0 next 2735
2021-08-09 03:52:38.113674: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d600 of size 256 by op assert_less_equal_29/Assert/AssertGuard/pivot_f/_825 action_count 94195480111501 step 0 next 2969
2021-08-09 03:52:38.113706: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d700 of size 256 by op assert_less_equal_29/Assert/AssertGuard/pivot_t/_826 action_count 94195480111502 step 0 next 2940
2021-08-09 03:52:38.113731: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d800 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_f/_843 action_count 94195480111503 step 0 next 2957
2021-08-09 03:52:38.113756: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69d900 of size 256 by op assert_greater_equal_30/Assert/AssertGuard/pivot_t/_844 action_count 94195480111504 step 0 next 2811
2021-08-09 03:52:38.113780: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69da00 of size 256 by op gradient_tape/unet_depth4/encode2/batch_normalization_584/moments/scalar action_count 94195480111505 step 0 nex
t 3012
2021-08-09 03:52:38.113805: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69db00 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_f/_853 action_count 94195480111506 step 0 next 2782
2021-08-09 03:52:38.113828: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69dc00 of size 256 by op assert_less_equal_30/Assert/AssertGuard/pivot_t/_854 action_count 94195480111507 step 0 next 2816
2021-08-09 03:52:38.113851: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69dd00 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_f/_871 action_count 94195480111508 step 0 next 1605
2021-08-09 03:52:38.113876: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69de00 of size 256 by op assert_greater_equal_31/Assert/AssertGuard/pivot_t/_872 action_count 94195480111509 step 0 next 2740
2021-08-09 03:52:38.113900: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69df00 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_f/_881 action_count 94195480111510 step 0 next 2942
2021-08-09 03:52:38.113923: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e000 of size 256 by op assert_less_equal_31/Assert/AssertGuard/pivot_t/_882 action_count 94195480111511 step 0 next 2905
2021-08-09 03:52:38.113957: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e100 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_f/_899 action_count 94195480111512 step 0 next 2806
2021-08-09 03:52:38.113981: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e200 of size 256 by op assert_greater_equal_32/Assert/AssertGuard/pivot_t/_900 action_count 94195480111513 step 0 next 2987
2021-08-09 03:52:38.114006: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e300 of size 256 by op gradient_tape/unet_depth4/encode1/batch_normalization_583/moments/scalar action_count 94195480111514 step 0 nex
t 2968
2021-08-09 03:52:38.114031: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e400 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_f/_909 action_count 94195480111515 step 0 next 3052
2021-08-09 03:52:38.114055: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e500 of size 256 by op assert_less_equal_32/Assert/AssertGuard/pivot_t/_910 action_count 94195480111516 step 0 next 3020
2021-08-09 03:52:38.114080: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e600 of size 256 by op gradient_tape/unet_depth4/encode1/batch_normalization_582/moments/scalar action_count 94195480111517 step 0 nex
t 2773
2021-08-09 03:52:38.114105: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e700 of size 256 by op gradient_tape/unet_depth4/encode0/batch_normalization_581/moments/scalar action_count 94195480111518 step 0 nex
t 2792
2021-08-09 03:52:38.114141: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e800 of size 256 by op gradient_tape/unet_depth4/encode0/batch_normalization_580/moments/scalar action_count 94195480111519 step 0 nex
t 2842
2021-08-09 03:52:38.114166: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69e900 of size 256 by op Adam/Adam/Const action_count 94195480111520 step 0 next 3023
2021-08-09 03:52:38.114200: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe69ea00 of size 1441792 by op gradient_tape/binary_ce_dice/logistic_loss/sub/Neg/_0__cf__4291 action_count 94195480111521 step 0 next 279
8
2021-08-09 03:52:38.114227: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe7fea00 of size 1441792 by op gradient_tape/binary_ce_dice/weighted_loss/Tile_1/_1__cf__4292 action_count 94195480111522 step 0 next 2948
2021-08-09 03:52:38.114252: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3afe95ea00 of size 157947392 by op Tile_90 action_count 94195480111525 step 13755587135625806913 next 18446744073709551615
2021-08-09 03:52:38.114277: I tensorflow/core/common_runtime/bfc_allocator.cc:1027] Next region of size 2147483648
2021-08-09 03:52:38.114304: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3b96000000 of size 340262912 by op Adam/gradients/AddN_3/tmp_var_zeros action_count 94195480111374 step 0 next 1663
2021-08-09 03:52:38.114340: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3baa480000 of size 170131456 by op unet_depth4/encode0/conv1d_506/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195480111553
 step 13755587135625806913 next 2699
2021-08-09 03:52:38.114370: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bb46c0000 of size 512 by op unet_depth4/encode0/batch_normalization_580/moments/variance action_count 94195480111559 step 13755587135625806
913 next 3044
2021-08-09 03:52:38.114396: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bb46c0200 of size 170131456 by op gradient_tape/unet_depth4/encode0/conv1d_507/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOpti
mizer action_count 94195480111571 step 13755587135625806913 next 2506
2021-08-09 03:52:38.114424: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bbe900200 of size 512 by op unet_depth4/encode0/batch_normalization_581/moments/mean action_count 94195480111593 step 13755587135625806913
next 2970
2021-08-09 03:52:38.114457: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bbe900400 of size 512 by op unet_depth4/encode0/batch_normalization_581/moments/variance action_count 94195480111597 step 13755587135625806
913 next 2492
2021-08-09 03:52:38.114485: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bbe900600 of size 512 by op unet_depth4/encode0/batch_normalization_581/batchnorm/mul action_count 94195480111605 step 13755587135625806913
 next 3019
2021-08-09 03:52:38.114512: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bbe900800 of size 170129920 by op unet_depth4/mp_encode0/MaxPool action_count 94195480111610 step 13755587135625806913 next 2951
2021-08-09 03:52:38.114545: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bc8b40200 of size 170131456 by op unet_depth4/encode0/conv1d_507/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195480111591
 step 13755587135625806913 next 3090
2021-08-09 03:52:38.114581: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bd2d80200 of size 170131456 by op unet_depth4/encode0/batch_normalization_581/batchnorm/mul_1 action_count 94195480111606 step 137555871356
25806913 next 2838
2021-08-09 03:52:38.114610: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bdcfc0200 of size 170131456 by op gradient_tape/unet_depth4/mp_encode0/MaxPool/MaxPoolGrad-0-TransposeNHWCToNCHW-LayoutOptimizer action_cou
nt 94195480111609 step 13755587135625806913 next 2829
2021-08-09 03:52:38.114638: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7200200 of size 1024 by op unet_depth4/encode1/batch_normalization_582/moments/mean action_count 94195480111635 step 13755587135625806913
 next 1051
2021-08-09 03:52:38.114663: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7200600 of size 1024 by op unet_depth4/encode1/batch_normalization_582/moments/variance action_count 94195480111639 step 1375558713562580
6913 next 2989
2021-08-09 03:52:38.114696: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7200a00 of size 1024 by op unet_depth4/encode1/batch_normalization_582/batchnorm/mul action_count 94195480111647 step 1375558713562580691
3 next 3040
2021-08-09 03:52:38.114724: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7200e00 of size 1024 by op unet_depth4/encode1/batch_normalization_583/moments/mean action_count 94195480111673 step 13755587135625806913
 next 2344
2021-08-09 03:52:38.114751: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7201200 of size 1024 by op unet_depth4/encode1/batch_normalization_583/moments/variance action_count 94195480111677 step 1375558713562580
6913 next 2890
2021-08-09 03:52:38.114778: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7201600 of size 1024 by op unet_depth4/encode1/batch_normalization_583/batchnorm/mul action_count 94195480111685 step 1375558713562580691
3 next 2930
2021-08-09 03:52:38.114802: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7201a00 of size 2048 by op unet_depth4/encode2/batch_normalization_584/moments/mean action_count 94195480111715 step 13755587135625806913
 next 3034
2021-08-09 03:52:38.114836: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7202200 of size 2048 by op unet_depth4/encode2/batch_normalization_584/moments/variance action_count 94195480111719 step 1375558713562580
6913 next 2704
2021-08-09 03:52:38.114862: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7202a00 of size 2048 by op unet_depth4/encode2/batch_normalization_584/batchnorm/mul action_count 94195480111727 step 1375558713562580691
3 next 3066
2021-08-09 03:52:38.114888: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7203200 of size 2048 by op unet_depth4/encode2/batch_normalization_585/moments/mean action_count 94195480111753 step 13755587135625806913
 next 1476
2021-08-09 03:52:38.114914: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7203a00 of size 2048 by op unet_depth4/encode2/batch_normalization_585/moments/variance action_count 94195480111757 step 1375558713562580
6913 next 3091
2021-08-09 03:52:38.114942: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7204200 of size 2048 by op unet_depth4/encode2/batch_normalization_585/batchnorm/mul action_count 94195480111765 step 1375558713562580691
3 next 2729
2021-08-09 03:52:38.114976: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7204a00 of size 2048 by op unet_depth4/encode3/batch_normalization_586/moments/mean action_count 94195480111795 step 13755587135625806913
 next 3041
2021-08-09 03:52:38.115002: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7205200 of size 2048 by op unet_depth4/encode3/batch_normalization_586/moments/variance action_count 94195480111797 step 1375558713562580
6913 next 3065
2021-08-09 03:52:38.115027: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7205a00 of size 2048 by op unet_depth4/encode3/batch_normalization_586/batchnorm/mul action_count 94195480111803 step 1375558713562580691
3 next 2971
2021-08-09 03:52:38.115052: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7206200 of size 2048 by op unet_depth4/encode3/batch_normalization_587/moments/mean action_count 94195480111829 step 13755587135625806913
 next 1680
2021-08-09 03:52:38.115076: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7206a00 of size 2048 by op unet_depth4/encode3/batch_normalization_587/moments/variance action_count 94195480111831 step 1375558713562580
6913 next 2880
2021-08-09 03:52:38.115112: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7207200 of size 2048 by op unet_depth4/encode3/batch_normalization_587/batchnorm/mul action_count 94195480111837 step 1375558713562580691
3 next 2712
2021-08-09 03:52:38.115140: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3be7207a00 of size 85035008 by op unet_depth4/mp_encode3/MaxPool action_count 94195480111842 step 13755587135625806913 next 2763
2021-08-09 03:52:38.115167: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bec320200 of size 85065728 by op gradient_tape/unet_depth4/encode1/conv1d_508/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOptim
izer action_count 94195480111612 step 13755587135625806913 next 2765
2021-08-09 03:52:38.115195: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bf1440200 of size 170131456 by op unet_depth4/encode1/batch_normalization_582/batchnorm/mul_1 action_count 94195480111648 step 137555871356
25806913 next 3086
2021-08-09 03:52:38.115220: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3bfb680200 of size 170131456 by op unet_depth4/encode1/conv1d_508/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195480111633
 step 13755587135625806913 next 2790
2021-08-09 03:52:38.115257: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c058c0200 of size 276037120 by op gradient_tape/unet_depth4/encode1/conv1d_509/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOpti
mizer action_count 94195480111651 step 13755587135625806913 next 18446744073709551615
2021-08-09 03:52:38.115285: I tensorflow/core/common_runtime/bfc_allocator.cc:1027] Next region of size 4294967296
2021-08-09 03:52:38.115309: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c36000000 of size 170131456 by op unet_depth4/encode1/batch_normalization_583/batchnorm/mul_1 action_count 94195480111686 step 137555871356
25806913 next 3009
2021-08-09 03:52:38.115334: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c40240000 of size 170131456 by op unet_depth4/encode1/conv1d_509/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195480111671
 step 13755587135625806913 next 2924
2021-08-09 03:52:38.115360: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c4a480000 of size 170131456 by op gradient_tape/unet_depth4/mp_encode1/MaxPool/MaxPoolGrad-0-TransposeNHWCToNCHW-LayoutOptimizer action_cou
nt 94195480111689 step 13755587135625806913 next 3072
2021-08-09 03:52:38.115388: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c546c0000 of size 85065728 by op unet_depth4/mp_encode1/MaxPool action_count 94195480111690 step 13755587135625806913 next 2768
2021-08-09 03:52:38.115423: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c597e0000 of size 85065728 by op unet_depth4/mp_encode2/MaxPool action_count 94195480111770 step 13755587135625806913 next 737
2021-08-09 03:52:38.115450: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c5e900000 of size 85065728 by op gradient_tape/unet_depth4/encode2/conv1d_510/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOptim
izer action_count 94195480111692 step 13755587135625806913 next 3006
2021-08-09 03:52:38.115479: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c63a20000 of size 170131456 by op unet_depth4/encode2/batch_normalization_584/batchnorm/mul_1 action_count 94195480111728 step 137555871356
25806913 next 3013
2021-08-09 03:52:38.115504: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c6dc60000 of size 170131456 by op unet_depth4/encode2/conv1d_510/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195480111713
 step 13755587135625806913 next 3035
2021-08-09 03:52:38.115548: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c77ea0000 of size 170131456 by op gradient_tape/unet_depth4/encode2/conv1d_511/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOpti
mizer action_count 94195480111731 step 13755587135625806913 next 2953
2021-08-09 03:52:38.115576: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c820e0000 of size 170131456 by op unet_depth4/encode2/batch_normalization_585/batchnorm/mul_1 action_count 94195480111766 step 137555871356
25806913 next 2995
2021-08-09 03:52:38.115602: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c8c320000 of size 170131456 by op unet_depth4/encode2/conv1d_511/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195480111751
 step 13755587135625806913 next 2966
2021-08-09 03:52:38.115628: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3c96560000 of size 170131456 by op gradient_tape/unet_depth4/mp_encode2/MaxPool/MaxPoolGrad-0-TransposeNHWCToNCHW-LayoutOptimizer action_cou
nt 94195480111769 step 13755587135625806913 next 2819
2021-08-09 03:52:38.115655: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a0000 of size 4096 by op unet_depth4/two_conv_center/batch_normalization_588/moments/mean action_count 94195480111867 step 137555871356
25806913 next 2956
2021-08-09 03:52:38.115680: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a1000 of size 4096 by op unet_depth4/two_conv_center/batch_normalization_588/moments/variance action_count 94195480111869 step 13755587
135625806913 next 2939
2021-08-09 03:52:38.115705: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a2000 of size 4096 by op unet_depth4/two_conv_center/batch_normalization_588/batchnorm/mul action_count 94195480111875 step 13755587135
625806913 next 2727
2021-08-09 03:52:38.115730: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a3000 of size 4096 by op unet_depth4/two_conv_center/batch_normalization_589/moments/mean action_count 94195480111901 step 137555871356
25806913 next 3085
2021-08-09 03:52:38.115767: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a4000 of size 4096 by op unet_depth4/two_conv_center/batch_normalization_589/moments/variance action_count 94195480111903 step 13755587
135625806913 next 3046
2021-08-09 03:52:38.115793: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a5000 of size 4096 by op unet_depth4/two_conv_center/batch_normalization_589/batchnorm/mul action_count 94195480111909 step 13755587135
625806913 next 2783
2021-08-09 03:52:38.115819: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a6000 of size 2048 by op unet_depth4/conv_transpose_decoder3/batch_normalization_590/moments/mean action_count 94195480111923 step 1375
5587135625806913 next 2980
2021-08-09 03:52:38.115844: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a6800 of size 2048 by op unet_depth4/conv_transpose_decoder3/batch_normalization_590/batchnorm/mul action_count 94195480111931 step 137
55587135625806913 next 2935
2021-08-09 03:52:38.115880: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a7000 of size 2048 by op unet_depth4/conv_transpose_decoder3/batch_normalization_590/moments/variance action_count 94195480111926 step
13755587135625806913 next 2919
2021-08-09 03:52:38.115908: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a7800 of size 2048 by op unet_depth4/two_conv_decoder3/batch_normalization_591/moments/mean action_count 94195480111959 step 1375558713
5625806913 next 3064
2021-08-09 03:52:38.115934: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a8000 of size 2048 by op unet_depth4/two_conv_decoder3/batch_normalization_591/batchnorm/mul action_count 94195480111967 step 137555871
35625806913 next 2746
2021-08-09 03:52:38.115962: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a8800 of size 2048 by op unet_depth4/two_conv_decoder3/batch_normalization_591/moments/variance action_count 94195480111962 step 137555
87135625806913 next 2778
2021-08-09 03:52:38.115988: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a9000 of size 2048 by op unet_depth4/two_conv_decoder3/batch_normalization_592/moments/mean action_count 94195480111979 step 1375558713
5625806913 next 3014
2021-08-09 03:52:38.116011: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07a9800 of size 2048 by op unet_depth4/two_conv_decoder3/batch_normalization_592/batchnorm/add action_count 94195480111985 step 137555871
35625806913 next 3005
2021-08-09 03:52:38.116045: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07aa000 of size 2048 by op unet_depth4/conv_transpose_decoder2/batch_normalization_593/moments/mean action_count 94195480112001 step 1375
5587135625806913 next 3038
2021-08-09 03:52:38.116073: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07aa800 of size 2048 by op unet_depth4/two_conv_decoder3/batch_normalization_592/batchnorm/mul action_count 94195480111986 step 137555871
35625806913 next 2885
2021-08-09 03:52:38.116106: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07ab000 of size 2048 by op unet_depth4/conv_transpose_decoder2/batch_normalization_593/batchnorm/mul action_count 94195480112009 step 137
55587135625806913 next 2738
2021-08-09 03:52:38.116135: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07ab800 of size 2048 by op unet_depth4/conv_transpose_decoder2/batch_normalization_593/moments/variance action_count 94195480112004 step
13755587135625806913 next 2695
2021-08-09 03:52:38.116161: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07ac000 of size 2048 by op unet_depth4/two_conv_decoder2/batch_normalization_594/moments/mean action_count 94195480112037 step 1375558713
5625806913 next 2869
2021-08-09 03:52:38.116186: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07ac800 of size 2048 by op unet_depth4/two_conv_decoder2/batch_normalization_594/batchnorm/mul action_count 94195480112045 step 137555871
35625806913 next 3054
2021-08-09 03:52:38.116210: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07ad000 of size 2048 by op unet_depth4/two_conv_decoder2/batch_normalization_594/moments/variance action_count 94195480112040 step 137555
87135625806913 next 2959
2021-08-09 03:52:38.116237: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07ad800 of size 2048 by op unet_depth4/two_conv_decoder2/batch_normalization_595/moments/mean action_count 94195480112071 step 1375558713
5625806913 next 2760
2021-08-09 03:52:38.116260: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07ae000 of size 2048 by op unet_depth4/two_conv_decoder2/batch_normalization_595/batchnorm/add action_count 94195480112077 step 137555871
35625806913 next 3031
2021-08-09 03:52:38.116296: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07ae800 of size 2048 by op unet_depth4/conv_transpose_decoder1/batch_normalization_596/moments/mean action_count 94195480112093 step 1375
5587135625806913 next 2972
2021-08-09 03:52:38.116323: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07af000 of size 2048 by op unet_depth4/two_conv_decoder2/batch_normalization_595/batchnorm/mul action_count 94195480112078 step 137555871
35625806913 next 2903
2021-08-09 03:52:38.116360: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07af800 of size 2048 by op unet_depth4/conv_transpose_decoder1/batch_normalization_596/batchnorm/mul action_count 94195480112105 step 137
55587135625806913 next 2788
2021-08-09 03:52:38.116387: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b0000 of size 2048 by op unet_depth4/conv_transpose_decoder1/batch_normalization_596/moments/variance action_count 94195480112098 step
13755587135625806913 next 3004
2021-08-09 03:52:38.116413: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b0800 of size 2048 by op unet_depth4/two_conv_decoder1/batch_normalization_597/moments/mean action_count 94195480112133 step 1375558713
5625806913 next 2706
2021-08-09 03:52:38.116438: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b1000 of size 2048 by op unet_depth4/two_conv_decoder1/batch_normalization_597/batchnorm/mul action_count 94195480112145 step 137555871
35625806913 next 3061
2021-08-09 03:52:38.116463: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b1800 of size 2048 by op unet_depth4/two_conv_decoder1/batch_normalization_597/moments/variance action_count 94195480112138 step 137555
87135625806913 next 2962
2021-08-09 03:52:38.116499: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b2000 of size 2048 by op unet_depth4/two_conv_decoder1/batch_normalization_598/moments/mean action_count 94195480112171 step 1375558713
5625806913 next 2889
2021-08-09 03:52:38.116533: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b2800 of size 2048 by op unet_depth4/two_conv_decoder1/batch_normalization_598/batchnorm/add action_count 94195480112181 step 137555871
35625806913 next 2926
2021-08-09 03:52:38.116561: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b3000 of size 1024 by op unet_depth4/conv_transpose_decoder0/batch_normalization_599/moments/mean action_count 94195480112197 step 1375
5587135625806913 next 3043
2021-08-09 03:52:38.116587: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b3400 of size 1024 by op unet_depth4/conv_transpose_decoder0/batch_normalization_599/batchnorm/mul action_count 94195480112209 step 137
55587135625806913 next 2955
2021-08-09 03:52:38.116622: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b3800 of size 2048 by op unet_depth4/two_conv_decoder1/batch_normalization_598/batchnorm/mul action_count 94195480112182 step 137555871
35625806913 next 3028
2021-08-09 03:52:38.116648: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca07b4000 of size 1024 by op unet_depth4/conv_transpose_decoder0/batch_normalization_599/moments/variance action_count 94195480112202 step
13755587135625806913 next 3081
2021-08-09 03:52:38.116675: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Free  at 2b3ca07b4400 of size 84982784 by op UNUSED action_count 94195480112212 step 13755587135625806913 next 1631
2021-08-09 03:52:38.116700: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ca58c0000 of size 85065728 by op gradient_tape/unet_depth4/encode3/conv1d_512/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOptim
izer action_count 94195480111772 step 13755587135625806913 next 2950
2021-08-09 03:52:38.116727: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3caa9e0000 of size 92274688 by op unet_depth4/encode3/batch_normalization_586/batchnorm/mul_1 action_count 94195480111804 step 1375558713562
5806913 next 3082
2021-08-09 03:52:38.116762: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cb01e0000 of size 92274688 by op unet_depth4/encode3/conv1d_512/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195480111793
step 13755587135625806913 next 412
2021-08-09 03:52:38.116790: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cb59e0000 of size 92274688 by op gradient_tape/unet_depth4/encode3/conv1d_513/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOptim
izer action_count 94195480111807 step 13755587135625806913 next 3053
2021-08-09 03:52:38.116817: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cbb1e0000 of size 92274688 by op unet_depth4/encode3/batch_normalization_587/batchnorm/mul_1 action_count 94195480111838 step 1375558713562
5806913 next 2988
2021-08-09 03:52:38.116842: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cc09e0000 of size 92274688 by op unet_depth4/encode3/conv1d_513/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195480111827
step 13755587135625806913 next 2982
2021-08-09 03:52:38.116869: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cc61e0000 of size 92274688 by op gradient_tape/unet_depth4/mp_encode3/MaxPool/MaxPoolGrad-0-TransposeNHWCToNCHW-LayoutOptimizer action_coun
t 94195480111841 step 13755587135625806913 next 2944
2021-08-09 03:52:38.116905: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ccb9e0000 of size 46137344 by op gradient_tape/unet_depth4/two_conv_center/conv1d_514/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-Lay
outOptimizer action_count 94195480111844 step 13755587135625806913 next 2964
2021-08-09 03:52:38.116933: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cce5e0000 of size 92274688 by op unet_depth4/two_conv_center/batch_normalization_588/batchnorm/mul_1 action_count 94195480111876 step 13755
587135625806913 next 2991
2021-08-09 03:52:38.116960: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cd3de0000 of size 92274688 by op unet_depth4/two_conv_center/conv1d_514/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 9419548
0111865 step 13755587135625806913 next 2719
2021-08-09 03:52:38.116986: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cd95e0000 of size 92274688 by op gradient_tape/unet_depth4/two_conv_center/conv1d_515/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-Lay
outOptimizer action_count 94195480111879 step 13755587135625806913 next 2694
2021-08-09 03:52:38.117012: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cdede0000 of size 92274688 by op unet_depth4/two_conv_center/batch_normalization_589/batchnorm/mul_1 action_count 94195480111910 step 13755
587135625806913 next 770
2021-08-09 03:52:38.117049: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ce45e0000 of size 92274688 by op unet_depth4/two_conv_center/conv1d_515/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 9419548
0111899 step 13755587135625806913 next 2979
2021-08-09 03:52:38.117077: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ce9de0000 of size 92274688 by op gradient_tape/unet_depth4/conv_transpose_decoder3/conv1d_transpose_104/conv1d_transpose/Conv2DBackpropFilt
er-2-TransposeNHWCToNCHW-LayoutOptimizer action_count 94195480111913 step 13755587135625806913 next 2999
2021-08-09 03:52:38.117104: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cef5e0000 of size 92274688 by op unet_depth4/conv_transpose_decoder3/conv1d_transpose_104/conv1d_transpose-0-0-TransposeNCHWToNHWC-LayoutOp
timizer action_count 94195480111921 step 13755587135625806913 next 2993
2021-08-09 03:52:38.117131: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cf4de0000 of size 92274688 by op unet_depth4/conv_transpose_decoder3/batch_normalization_590/batchnorm/mul_1 action_count 94195480111933 st
ep 13755587135625806913 next 3071
2021-08-09 03:52:38.117167: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cfa5e0000 of size 92274688 by op unet_depth4/two_conv_decoder3/batch_normalization_591/batchnorm/mul_1 action_count 94195480111969 step 137
55587135625806913 next 3001
2021-08-09 03:52:38.117195: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3cffde0000 of size 92274688 by op unet_depth4/two_conv_decoder3/conv1d_516/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195
480111957 step 13755587135625806913 next 2908
2021-08-09 03:52:38.117223: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d055e0000 of size 184549376 by op gradient_tape/unet_depth4/two_conv_decoder3/conv1d_516/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-
LayoutOptimizer action_count 94195480111936 step 13755587135625806913 next 2977
2021-08-09 03:52:38.117251: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d105e0000 of size 92274688 by op gradient_tape/unet_depth4/two_conv_decoder3/conv1d_517/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-L
ayoutOptimizer action_count 94195480111971 step 13755587135625806913 next 2736
2021-08-09 03:52:38.117278: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d15de0000 of size 92274688 by op unet_depth4/two_conv_decoder3/batch_normalization_592/batchnorm/mul_1 action_count 94195480111989 step 137
55587135625806913 next 2910
2021-08-09 03:52:38.117304: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d1b5e0000 of size 92274688 by op unet_depth4/two_conv_decoder3/conv1d_517/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 94195
480111977 step 13755587135625806913 next 2685
2021-08-09 03:52:38.117331: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d20de0000 of size 92274688 by op gradient_tape/unet_depth4/conv_transpose_decoder2/conv1d_transpose_105/conv1d_transpose/Conv2DBackpropFilt
er-2-TransposeNHWCToNCHW-LayoutOptimizer action_count 94195480111991 step 13755587135625806913 next 3073
2021-08-09 03:52:38.117358: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d265e0000 of size 262275072 by op unet_depth4/conv_transpose_decoder2/conv1d_transpose_105/conv1d_transpose-0-0-TransposeNCHWToNHWC-LayoutO
ptimizer action_count 94195480111999 step 13755587135625806913 next 18446744073709551615
2021-08-09 03:52:38.117392: I tensorflow/core/common_runtime/bfc_allocator.cc:1027] Next region of size 6764888064
2021-08-09 03:52:38.117420: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d70000000 of size 184549376 by op unet_depth4/conv_transpose_decoder2/batch_normalization_593/batchnorm/mul_1 action_count 94195480112011 s
tep 13755587135625806913 next 3032
2021-08-09 03:52:38.117447: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d7b000000 of size 184549376 by op unet_depth4/two_conv_decoder2/batch_normalization_594/batchnorm/mul_1 action_count 94195480112047 step 13
755587135625806913 next 2626
2021-08-09 03:52:38.117473: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Free  at 2b3d86000000 of size 170131456 by op UNUSED action_count 94195421079702 step 17448270550709606267 next 2997
2021-08-09 03:52:38.117499: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3d90240000 of size 354680832 by op gradient_tape/unet_depth4/two_conv_decoder2/conv1d_518/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-
LayoutOptimizer action_count 94195480112014 step 13755587135625806913 next 2883
2021-08-09 03:52:38.117543: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3da5480000 of size 184549376 by op unet_depth4/two_conv_decoder2/conv1d_518/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 9419
5480112035 step 13755587135625806913 next 3010
2021-08-09 03:52:38.117571: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3db0480000 of size 184549376 by op gradient_tape/unet_depth4/two_conv_decoder2/conv1d_519/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-
LayoutOptimizer action_count 94195480112049 step 13755587135625806913 next 2943
2021-08-09 03:52:38.117597: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3dbb480000 of size 184549376 by op unet_depth4/two_conv_decoder2/batch_normalization_595/batchnorm/mul_1 action_count 94195480112081 step 13
755587135625806913 next 2973
2021-08-09 03:52:38.117624: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3dc6480000 of size 184549376 by op unet_depth4/two_conv_decoder2/conv1d_519/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 9419
5480112069 step 13755587135625806913 next 2984
2021-08-09 03:52:38.117650: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3dd1480000 of size 184549376 by op gradient_tape/unet_depth4/conv_transpose_decoder1/conv1d_transpose_106/conv1d_transpose/Conv2DBackpropFil
ter-2-TransposeNHWCToNCHW-LayoutOptimizer action_count 94195480112083 step 13755587135625806913 next 2946
2021-08-09 03:52:38.117677: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ddc480000 of size 340262912 by op unet_depth4/conv_transpose_decoder1/conv1d_transpose_106/conv1d_transpose-0-0-TransposeNCHWToNHWC-LayoutO
ptimizer action_count 94195480112091 step 13755587135625806913 next 2002
2021-08-09 03:52:38.117713: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3df0900000 of size 340262912 by op unet_depth4/conv_transpose_decoder1/batch_normalization_596/batchnorm/mul_1 action_count 94195480112107 s
tep 13755587135625806913 next 3036
2021-08-09 03:52:38.117741: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3e04d80000 of size 340262912 by op unet_depth4/two_conv_decoder1/batch_normalization_597/batchnorm/mul_1 action_count 94195480112147 step 13
755587135625806913 next 3062
2021-08-09 03:52:38.117777: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Free  at 2b3e19200000 of size 170131456 by op UNUSED action_count 94195419013976 step 17479529273785244716 next 2934
2021-08-09 03:52:38.117806: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3e23440000 of size 510394368 by op gradient_tape/unet_depth4/two_conv_decoder1/conv1d_520/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-
LayoutOptimizer action_count 94195480112110 step 13755587135625806913 next 3033
2021-08-09 03:52:38.117832: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3e41b00000 of size 340262912 by op unet_depth4/two_conv_decoder1/conv1d_520/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 9419
5480112131 step 13755587135625806913 next 2927
2021-08-09 03:52:38.117861: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3e55f80000 of size 340262912 by op gradient_tape/unet_depth4/two_conv_decoder1/conv1d_521/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-
LayoutOptimizer action_count 94195480112149 step 13755587135625806913 next 3022
2021-08-09 03:52:38.117889: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3e6a400000 of size 340262912 by op unet_depth4/two_conv_decoder1/batch_normalization_598/batchnorm/mul_1 action_count 94195480112185 step 13
755587135625806913 next 2998
2021-08-09 03:52:38.117922: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3e7e880000 of size 340262912 by op unet_depth4/two_conv_decoder1/conv1d_521/conv1d-0-0-TransposeNCHWToNHWC-LayoutOptimizer action_count 9419
5480112169 step 13755587135625806913 next 3029
2021-08-09 03:52:38.117951: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3e92d00000 of size 340262912 by op gradient_tape/unet_depth4/conv_transpose_decoder0/conv1d_transpose_107/conv1d_transpose/Conv2DBackpropFil
ter-2-TransposeNHWCToNCHW-LayoutOptimizer action_count 94195480112187 step 13755587135625806913 next 2938
2021-08-09 03:52:38.117978: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ea7180000 of size 340262912 by op unet_depth4/conv_transpose_decoder0/conv1d_transpose_107/conv1d_transpose-0-0-TransposeNCHWToNHWC-LayoutO
ptimizer action_count 94195480112195 step 13755587135625806913 next 2945
2021-08-09 03:52:38.118005: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ebb600000 of size 340262912 by op unet_depth4/conv_transpose_decoder0/batch_normalization_599/batchnorm/mul_1 action_count 94195480112211 s
tep 13755587135625806913 next 3021
2021-08-09 03:52:38.118031: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] InUse at 2b3ecfa80000 of size 510394368 by op unet_depth4/decoder0/concat action_count 94195480112213 step 13755587135625806913 next 3080
2021-08-09 03:52:38.118055: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Free  at 2b3eee140000 of size 354680832 by op UNUSED action_count 94195419013931 step 17479529273785244716 next 18446744073709551615
2021-08-09 03:52:38.118090: I tensorflow/core/common_runtime/bfc_allocator.cc:1051]      Summary of in-use Chunks by size:
2021-08-09 03:52:38.118121: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1712 Chunks of size 256 totalling 428.0KiB
2021-08-09 03:52:38.118149: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 140 Chunks of size 512 totalling 70.0KiB
2021-08-09 03:52:38.118185: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 66 Chunks of size 768 totalling 49.5KiB
2021-08-09 03:52:38.118212: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 167 Chunks of size 1024 totalling 167.0KiB
2021-08-09 03:52:38.118237: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 64 Chunks of size 1280 totalling 80.0KiB
2021-08-09 03:52:38.118262: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 16 Chunks of size 1536 totalling 24.0KiB
2021-08-09 03:52:38.118290: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 95 Chunks of size 1792 totalling 166.2KiB
2021-08-09 03:52:38.118326: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 317 Chunks of size 2048 totalling 634.0KiB
2021-08-09 03:52:38.118354: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 11 Chunks of size 2304 totalling 24.8KiB
2021-08-09 03:52:38.118377: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 5 Chunks of size 2560 totalling 12.5KiB
2021-08-09 03:52:38.118402: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 8 Chunks of size 2816 totalling 22.0KiB
2021-08-09 03:52:38.118437: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 5 Chunks of size 3072 totalling 15.0KiB
2021-08-09 03:52:38.118463: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 6 Chunks of size 3328 totalling 19.5KiB
2021-08-09 03:52:38.118488: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 28 Chunks of size 3584 totalling 98.0KiB
2021-08-09 03:52:38.118511: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 5 Chunks of size 3840 totalling 18.8KiB
2021-08-09 03:52:38.118554: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 54 Chunks of size 4096 totalling 216.0KiB
2021-08-09 03:52:38.118581: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 4352 totalling 4.2KiB
2021-08-09 03:52:38.118605: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 4 Chunks of size 4608 totalling 18.0KiB
2021-08-09 03:52:38.118639: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 4864 totalling 9.5KiB
2021-08-09 03:52:38.118666: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 5632 totalling 5.5KiB
2021-08-09 03:52:38.118693: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 6656 totalling 13.0KiB
2021-08-09 03:52:38.118716: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 13312 totalling 39.0KiB
2021-08-09 03:52:38.118752: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 26368 totalling 25.8KiB
2021-08-09 03:52:38.118782: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 29696 totalling 29.0KiB
2021-08-09 03:52:38.118812: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 31488 totalling 30.8KiB
2021-08-09 03:52:38.118849: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 4 Chunks of size 52480 totalling 205.0KiB
2021-08-09 03:52:38.118878: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 69888 totalling 136.5KiB
2021-08-09 03:52:38.118904: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 77056 totalling 75.2KiB
2021-08-09 03:52:38.118943: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 78592 totalling 230.2KiB
2021-08-09 03:52:38.118971: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 15 Chunks of size 81920 totalling 1.17MiB
2021-08-09 03:52:38.118999: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 96512 totalling 94.2KiB
2021-08-09 03:52:38.119035: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 100352 totalling 98.0KiB
2021-08-09 03:52:38.119063: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 104704 totalling 306.8KiB
2021-08-09 03:52:38.119091: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 137472 totalling 268.5KiB
2021-08-09 03:52:38.119127: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 139264 totalling 136.0KiB
2021-08-09 03:52:38.119155: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 167168 totalling 489.8KiB
2021-08-09 03:52:38.119183: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 171008 totalling 167.0KiB
2021-08-09 03:52:38.119219: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 190208 totalling 185.8KiB
2021-08-09 03:52:38.119247: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 205568 totalling 200.8KiB
2021-08-09 03:52:38.119276: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 6 Chunks of size 209152 totalling 1.20MiB
2021-08-09 03:52:38.119308: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 270336 totalling 264.0KiB
2021-08-09 03:52:38.119334: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 4 Chunks of size 274944 totalling 1.05MiB
2021-08-09 03:52:38.119360: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 278784 totalling 816.8KiB
2021-08-09 03:52:38.119396: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 313856 totalling 613.0KiB
2021-08-09 03:52:38.119424: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 334336 totalling 979.5KiB
2021-08-09 03:52:38.119450: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 360448 totalling 704.0KiB
2021-08-09 03:52:38.119487: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 397824 totalling 388.5KiB
2021-08-09 03:52:38.119514: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 418304 totalling 1.20MiB
2021-08-09 03:52:38.119557: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 32 Chunks of size 425984 totalling 13.00MiB
2021-08-09 03:52:38.119582: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 9 Chunks of size 549632 totalling 4.72MiB
2021-08-09 03:52:38.119607: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 549888 totalling 537.0KiB
2021-08-09 03:52:38.119642: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 557568 totalling 544.5KiB
2021-08-09 03:52:38.119667: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 7 Chunks of size 668416 totalling 4.46MiB
2021-08-09 03:52:38.119702: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 824320 totalling 805.0KiB
2021-08-09 03:52:38.119728: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 4 Chunks of size 824576 totalling 3.15MiB
2021-08-09 03:52:38.119751: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 4 Chunks of size 836352 totalling 3.19MiB
2021-08-09 03:52:38.119786: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 891136 totalling 1.70MiB
2021-08-09 03:52:38.119815: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 994560 totalling 971.2KiB
2021-08-09 03:52:38.119839: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 995328 totalling 972.0KiB
2021-08-09 03:52:38.119872: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 1002752 totalling 1.91MiB
2021-08-09 03:52:38.119898: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 1054720 totalling 1.01MiB
2021-08-09 03:52:38.119923: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 1055488 totalling 1.01MiB
2021-08-09 03:52:38.119959: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 1081344 totalling 2.06MiB
2021-08-09 03:52:38.119985: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 5 Chunks of size 1099264 totalling 5.24MiB
2021-08-09 03:52:38.120012: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 1206016 totalling 1.15MiB
2021-08-09 03:52:38.120035: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 1254656 totalling 3.59MiB
2021-08-09 03:52:38.120072: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 1336832 totalling 3.82MiB
2021-08-09 03:52:38.120100: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 13 Chunks of size 1441792 totalling 17.88MiB
2021-08-09 03:52:38.120127: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 1532928 totalling 1.46MiB
2021-08-09 03:52:38.120163: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 1622016 totalling 3.09MiB
2021-08-09 03:52:38.120192: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 1933312 totalling 3.69MiB
2021-08-09 03:52:38.120218: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 1964800 totalling 1.87MiB
2021-08-09 03:52:38.120255: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 7 Chunks of size 2097152 totalling 14.00MiB
2021-08-09 03:52:38.120283: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 9 Chunks of size 2198272 totalling 18.87MiB
2021-08-09 03:52:38.120310: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 6 Chunks of size 2673408 totalling 15.30MiB
2021-08-09 03:52:38.120345: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 2723840 totalling 2.60MiB
2021-08-09 03:52:38.120373: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 2899968 totalling 8.30MiB
2021-08-09 03:52:38.120400: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 6 Chunks of size 2930944 totalling 16.77MiB
2021-08-09 03:52:38.120437: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 22 Chunks of size 3145728 totalling 66.00MiB
2021-08-09 03:52:38.120465: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 6 Chunks of size 3297536 totalling 18.87MiB
2021-08-09 03:52:38.120491: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 3714816 totalling 3.54MiB
2021-08-09 03:52:38.120534: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 4010240 totalling 11.47MiB
2021-08-09 03:52:38.120560: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 4046080 totalling 3.86MiB
2021-08-09 03:52:38.120586: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 5 Chunks of size 4194304 totalling 20.00MiB
2021-08-09 03:52:38.120621: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 4396544 totalling 8.39MiB
2021-08-09 03:52:38.120647: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 4673536 totalling 4.46MiB
2021-08-09 03:52:38.120672: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 4767744 totalling 13.64MiB
2021-08-09 03:52:38.120696: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 4894464 totalling 4.67MiB
2021-08-09 03:52:38.120728: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 5318400 totalling 5.07MiB
2021-08-09 03:52:38.120755: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 5426944 totalling 5.17MiB
2021-08-09 03:52:38.120780: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 5647360 totalling 5.39MiB
2021-08-09 03:52:38.120805: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 5 Chunks of size 5775360 totalling 27.54MiB
2021-08-09 03:52:38.120841: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 3 Chunks of size 6045696 totalling 17.30MiB
2021-08-09 03:52:38.120866: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 6225920 totalling 5.94MiB
2021-08-09 03:52:38.120891: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 12 Chunks of size 6291456 totalling 72.00MiB
2021-08-09 03:52:38.120925: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 6381568 totalling 6.09MiB
2021-08-09 03:52:38.120953: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 6754048 totalling 6.44MiB
2021-08-09 03:52:38.120978: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 6 Chunks of size 7012352 totalling 40.12MiB
2021-08-09 03:52:38.121003: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 7193600 totalling 6.86MiB
2021-08-09 03:52:38.121036: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 7728640 totalling 7.37MiB
2021-08-09 03:52:38.121062: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 8192000 totalling 15.62MiB
2021-08-09 03:52:38.121088: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 4 Chunks of size 8792832 totalling 33.54MiB
2021-08-09 03:52:38.121111: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 9883392 totalling 9.42MiB
2021-08-09 03:52:38.121146: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 10772736 totalling 10.27MiB
2021-08-09 03:52:38.121174: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 4 Chunks of size 12582912 totalling 48.00MiB
2021-08-09 03:52:38.121199: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 13459968 totalling 12.84MiB
2021-08-09 03:52:38.121232: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 5 Chunks of size 14024704 totalling 66.88MiB
2021-08-09 03:52:38.121259: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 14162944 totalling 13.51MiB
2021-08-09 03:52:38.121288: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 18874368 totalling 18.00MiB
2021-08-09 03:52:38.121323: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 24661504 totalling 23.52MiB
2021-08-09 03:52:38.121350: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 36044800 totalling 68.75MiB
2021-08-09 03:52:38.121373: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 46137344 totalling 44.00MiB
2021-08-09 03:52:38.121406: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 78643200 totalling 75.00MiB
2021-08-09 03:52:38.121434: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 85035008 totalling 81.10MiB
2021-08-09 03:52:38.121473: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 5 Chunks of size 85065728 totalling 405.62MiB
2021-08-09 03:52:38.121506: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 20 Chunks of size 92274688 totalling 1.72GiB
2021-08-09 03:52:38.121535: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 143653632 totalling 137.00MiB
2021-08-09 03:52:38.121560: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 157947392 totalling 150.63MiB
2021-08-09 03:52:38.121594: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 170129920 totalling 162.25MiB
2021-08-09 03:52:38.121619: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 16 Chunks of size 170131456 totalling 2.54GiB
2021-08-09 03:52:38.121646: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 8 Chunks of size 184549376 totalling 1.38GiB
2021-08-09 03:52:38.121671: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 245764096 totalling 234.38MiB
2021-08-09 03:52:38.121706: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 262275072 totalling 250.12MiB
2021-08-09 03:52:38.121732: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 276037120 totalling 263.25MiB
2021-08-09 03:52:38.121757: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 314572800 totalling 600.00MiB
2021-08-09 03:52:38.121789: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 11 Chunks of size 340262912 totalling 3.49GiB
2021-08-09 03:52:38.121817: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 1 Chunks of size 354680832 totalling 338.25MiB
2021-08-09 03:52:38.121844: I tensorflow/core/common_runtime/bfc_allocator.cc:1054] 2 Chunks of size 510394368 totalling 973.50MiB
2021-08-09 03:52:38.121879: I tensorflow/core/common_runtime/bfc_allocator.cc:1058] Sum Total of in-use chunks: 13.57GiB
2021-08-09 03:52:38.121906: I tensorflow/core/common_runtime/bfc_allocator.cc:1060] total_region_allocated_bytes_: 15354822656 memory_limit_: 15354822656 available bytes: 0 curr_region_allocation_bytes_: 17179869184
2021-08-09 03:52:38.121937: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Stats:
Limit:                     15354822656
InUse:                     14574896128
MaxInUse:                  14574896128
NumAllocs:                   225920650
MaxAllocSize:               3306898944
Reserved:                            0
PeakReserved:                        0
LargestFreeBlock:                    0

2021-08-09 03:52:38.122399: W tensorflow/core/common_runtime/bfc_allocator.cc:467] **************************************************************************************************__
2021-08-09 03:52:38.122475: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at transpose_op.cc:184 : Resource exhausted: OOM when allocating tensor with shape[22,354,1,16384] and type float on /job:localhost/replica:0
/task:0/device:GPU:0 by allocator GPU_0_bfc
Traceback (most recent call last):
  File "/beegfs/ye53nis/drmed-git/src/fluotracify/training/search_hparams.py", line 446, in <module>
    hparams_run()
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/beegfs/ye53nis/drmed-git/src/fluotracify/training/search_hparams.py", line 405, in hparams_run
    best_auc_val = run_one(
  File "/beegfs/ye53nis/drmed-git/src/fluotracify/training/search_hparams.py", line 306, in run_one
    result = model.fit(
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/mlflow/utils/autologging_utils/safety.py", line 490, in safe_patch_function
    patch_function.call(call_original, *args, **kwargs)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/mlflow/utils/autologging_utils/safety.py", line 156, in call
    return cls().__call__(original, *args, **kwargs)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/mlflow/utils/autologging_utils/safety.py", line 167, in __call__
    raise e
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/mlflow/utils/autologging_utils/safety.py", line 160, in __call__
    return self._patch_implementation(original, *args, **kwargs)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/mlflow/utils/autologging_utils/safety.py", line 218, in _patch_implementation
    result = super(PatchWithManagedRun, self)._patch_implementation(
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/mlflow/tensorflow.py", line 1097, in _patch_implementation
    history = original(inst, *args, **kwargs)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/mlflow/utils/autologging_utils/safety.py", line 448, in call_original
    original_result = original(*og_args, **og_kwargs)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/tensorflow/python/keras/engine/training.py", line 1183, in fit
    tmp_logs = self.train_function(iterator)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 950, in _call
    return self._stateless_fn(*args, **kwds)
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 3023, in __call__
    return graph_function._call_flat(
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 1960, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 591, in call
    outputs = execute.execute(
  File "/home/ye53nis/.conda/envs/mlflow-eaf130b8edd83d20d4f1e0db4286dabd625893fe/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
  (0) Resource exhausted:  OOM when allocating tensor with shape[22,354,1,16384] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
         [[{{node gradient_tape/unet_depth4/two_conv_decoder0/conv1d_522/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOptimizer}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

         [[assert_less_equal_4/Assert/AssertGuard/pivot_f/_125/_159]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

  (1) Resource exhausted:  OOM when allocating tensor with shape[22,354,1,16384] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
         [[{{node gradient_tape/unet_depth4/two_conv_decoder0/conv1d_522/conv1d/Conv2DBackpropFilter-0-TransposeNHWCToNCHW-LayoutOptimizer}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_65983138]

Function call stack:
train_function -> train_function

2021/08/09 03:52:57 ERROR mlflow.cli: === Run (ID 'b6cbd5623eb44c6c8158c97b7a40c651') failed ===
(tf) [ye53nis@node130 drmed-git]$

2.1.4 Analyze run 1 and 2

  1. start 
    %cd /beegfs/ye53nis/drmed-git

    /beegfs/ye53nis/drmed-git

    import mlflow
import sys

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

    sys.path.append('src/')
from fluotracify.training import build_model as bm, preprocess_data as ppd
from fluotracify.applications import correlate, plots, correction
from fluotracify.imports import ptu_utils as ptu
  2. Out of convenience, I used Mlflowui to compare the two runs with all parameters and metrics. Since a nice exporting option was missing, I copied the comparison into libreoffice calc by hand and saved it as a csv. Now lets load it and print the validation AUC as an example. 
    run1_2 = pd.read_csv('data/exp-210807-hparams/run1-2_comparison.csv', index_col=0)
run1_2_valauc = run1_2.loc['val_auc'].astype(float)
run1_2_valauc

    2761cb4ad23244beafac36189b805c3a    0.975
d92021ec45f5415283a8e5643c7e3449    0.975
e6b70d64227f42bcb647289d2af2afb4    0.944
876fa57de00643b1902f3a0be74e0682    0.940
29905abbb90344798ee59f1f40775880    0.929
                                    ...
158bfabc0fcc448dada729967f924574    0.969
3423b9fa10e44eb7b2cffd78596a193f    0.897
985ec48a8e1b4101abb8a153cee69b57    0.891
5c3e5090c67f4340bf207f949e925758    0.601
ff0bc51f1cd3413bbdebb97b0e127e75    0.889
Name: val_auc, Length: 75, dtype: float64
  3. I ran each hparams run twice for each random parameter sample. As a minimal statistical move, lets take the average of the metrics which have 2 identical parameter runs. Unfortunately, one run is only single, since the program crashed. We have to remove it and later add it again. 
    singles_ls = ['5441e71efe0f4dae868648e7cc795c65']
run1_2_singles = run1_2.loc[:, singles_ls]
run1_2_singles.iloc[35:, :] = run1_2_singles.iloc[35:, :].astype(np.float64)
run1_2_singles
    5441e71efe0f4dae868648e7cc795c65
    Run ID:
    Run Name: NaN
    Start Time: 2021-08-08 13:46:29
    batch_size None
    class_weight None
    epochs 20
    ... ...
    val_tp0.1 756740.0
    val_tp0.3 686440.0
    val_tp0.5 622523.0
    val_tp0.7 505054.0
    val_tp0.9 241926.0

    104 rows × 1 columns

    run1_2 = run1_2.drop(columns=singles_ls)

    assert len(run1_2.iloc[35:, :].columns) % 2 == 0

    run1_2_doubleparams = pd.DataFrame()
run1_2_doublemetrics = pd.DataFrame()
double_cols = []
for left, right in zip(run1_2.iloc[:, ::2].items(), run1_2.iloc[:, 1::2].items()):
    double_cols.append((left[0], right[0]))
    current_metrics = left[1].iloc[35:].combine(other=right[1].iloc[35:],
                                                func=(lambda x1, x2: (float(x1) + float(x2)) / 2))
    current_params = left[1].iloc[:35].combine(other=right[1].iloc[:35],
                                               func=(lambda x1, x2: set((x1, x2)) if x1 != x2 else x1))
    run1_2_doubleparams = pd.concat([run1_2_doubleparams, current_params], axis=1)
    run1_2_doublemetrics = pd.concat([run1_2_doublemetrics, current_metrics], axis=1)

run1_2_doublemetrics = pd.DataFrame(data=run1_2_doublemetrics.to_numpy(),
                                    index=run1_2.iloc[35:, :].index,
                                    columns=double_cols)

run1_2_doubleparams = pd.DataFrame(data=run1_2_doubleparams.to_numpy(),
                                   index=run1_2.iloc[:35, :].index,
                                   columns=double_cols)

run1_2_combimetrics = pd.concat([run1_2_doublemetrics, run1_2_singles.iloc[35:, :]], axis=1)
run1_2_combiparams = pd.concat([run1_2_doubleparams, run1_2_singles.iloc[:35, :]], axis=1)
run1_2_combiparams
    (2761cb4ad23244beafac36189b805c3a, d92021ec45f5415283a8e5643c7e3449) (e6b70d64227f42bcb647289d2af2afb4, 876fa57de00643b1902f3a0be74e0682) (29905abbb90344798ee59f1f40775880, 5a89360285384c728e9e06421ab97c8d) (ab0ff6fff89c4db6983aa98c7eee9663, b9d383f0e9ef4755a801376130594f2a) (481bacb91dcb4d63b16d0a8fe5d198ec, 2ef6df0c95b34b338c4b930f41695ed0) (d36564aa214c41bc966fc7b890df31a8, ba34b505ba324f208d428bd7797233e7) (0a484d3116ab436d8b4cf4a938d470c0, 5a2dfcd776014637bdaef2da3ab1d132) (b72a292b214c462480cbe2e66e811c78, 50c7e9bba0534a27801143bda2e88d35) (9051e32b87d84f3485b980067addec30, 61ff87bdb89b4e2ba64f8dacc774992d) (93b168c0ff7942c8a908a94129daf973, f243b3b742de4dbcb7ccfbd4244706f8) ... (3cbd945b62ec4634839372e403f6f377, 458b36a70db843719d202a8eda448f17) (2648b86e7ef54a0e9f7b57340703150e, 0aae5a802f5342cebed9e51839d48b41) (7719d9fbef8f4c22b710caf91cd527d2, 2b7aab186fad48158d53b78610550d73) (5bbaee372809482ea6d2d985b554cc4b, b8200ec9fb884c12b390e1a5b20299bc) (37ab5992ecdf4760a6292bc64e61e309, 91e4770c10ac4cceb405bfdcebf7c966) (90a949129948495fb7f3b955e8258461, ce09d20be06745fb84c9ff177c8a9cef) (d320b958bde94b1e998d210fd2e53efa, 158bfabc0fcc448dada729967f924574) (3423b9fa10e44eb7b2cffd78596a193f, 985ec48a8e1b4101abb8a153cee69b57) (5c3e5090c67f4340bf207f949e925758, ff0bc51f1cd3413bbdebb97b0e127e75) 5441e71efe0f4dae868648e7cc795c65
    Run ID:
    Run Name: {nan} {nan} {nan} {nan} {nan} {nan} {nan} {nan} {nan} {nan} ... {nan} {nan} {nan} {nan} {nan} {nan} {nan} {nan} {nan} NaN
    Start Time: {2021-08-09 03:35:27, 2021-08-09 03:18:38} {2021-08-09 03:00:40, 2021-08-09 02:43:08} {2021-08-09 02:21:19, 2021-08-09 01:59:18} {2021-08-09 01:45:12, 2021-08-09 01:52:15} {2021-08-09 01:01:03, 2021-08-09 00:16:49} {2021-08-09 00:01:24, 2021-08-08 23:46:06} {2021-08-08 23:09:26, 2021-08-08 23:27:50} {2021-08-08 23:00:25, 2021-08-08 22:51:22} {2021-08-08 22:10:33, 2021-08-08 21:29:38} {2021-08-08 21:15:24, 2021-08-08 21:01:49} ... {2021-08-08 04:30:30, 2021-08-08 04:19:31} {2021-08-08 03:58:55, 2021-08-08 03:38:13} {2021-08-08 03:12:28, 2021-08-08 03:25:26} {2021-08-08 03:05:58, 2021-08-08 02:59:20} {2021-08-08 02:04:14, 2021-08-08 01:12:38} {2021-08-08 01:05:46, 2021-08-08 00:58:39} {2021-08-08 00:26:43, 2021-08-08 00:43:13} {2021-08-07 23:57:58, 2021-08-07 23:31:44} {2021-08-07 22:57:43, 2021-08-07 22:23:42} 2021-08-08 13:46:29
    batch_size None None None None None None None None None None ... None None None None None None None None None None
    class_weight None None None None None None None None None None ... None None None None None None None None None None
    epochs 20 20 20 20 20 20 20 20 20 20 ... 20 20 20 20 20 20 20 20 20 20
    hp_batch_size 13 5 11 23 13 13 4 18 26 15 ... 9 4 18 10 4 20 14 5 18 14
    hp_epochs 20 20 20 20 20 20 20 20 20 20 ... 20 20 20 20 20 20 20 20 20 20
    hp_first_filters 107 33 102 58 108 41 64 59 44 23 ... 64 128 32 16 64 32 128 64 32 16
    hp_input_size 8192 4096 16384 8192 8192 16384 4096 4096 16384 16384 ... 4096 4096 16384 4096 8192 4096 4096 16384 16384 4096
    hp_lr_power 5 7 6 6 6 6 2 2 1 7 ... 1 5 5 1 1 5 1 5 1 5
    hp_lr_start 0.00782433731433605 0.0183838908333744 0.0482033274205359 0.00878855796222036 0.00652659278503235 0.0571175071464742 0.047837465592254 0.0330458251501699 0.0136170138242663 0.0305060808685107 ... 0.0100697459464075 0.024311131849965 0.0275283294963226 0.0923388677844686 0.0183950335666331 0.00805377739706802 0.0730305548495589 0.0100791137947559 0.0799422633823613 0.0271446293545328
    hp_n_levels 3 5 1 1 6 3 5 1 7 6 ... 5 3 5 3 9 5 3 3 7 7
    hp_pool_size 8 2 8 2 2 4 4 8 2 4 ... 4 4 4 2 2 4 4 4 2 2
    hp_scaler maxabs robust quant_g minmax minmax l1 standard quant_g standard quant_g ... maxabs maxabs maxabs minmax quant_g l2 robust robust l2 quant_g
    initial_epoch 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
    lr schedule [0.00782433731433605, 0.0060543230624031785, 0... [0.01838389083337442, 0.012838156616261365, 0.... [0.04820332742053588, 0.035433875087977615, 0.... [0.00878855796222036, 0.0064603976883159595, 0... [0.0065265927850323546, 0.004797645429688916, ... [0.057117507146474185, 0.041986616316088646, 0... [0.04783746559225395, 0.043173312697009185, 0.... [0.03304582515016993, 0.029823857198028363, 0.... [0.01361701382426631, 0.012936163133052994, 0.... [0.030506080868510657, 0.021303534028133002, 0... ... [0.010069745946407459, 0.009566258649087086, 0... [0.024311131849965008, 0.018811490394552027, 0... [0.02752832949632261, 0.021300896605473404, 0.... [0.0923388677844686, 0.08772192439524518, 0.08... [0.018395033566633148, 0.01747528188830149, 0.... [0.008053777397068015, 0.0062318594247195965, ... [0.07303055484955895, 0.06937902710708099, 0.0... [0.010079113794755897, 0.007799026121275398, 0... [0.0799422633823613, 0.07594515021324323, 0.07... [0.027144629354532844, 0.021003996750040436, 0...
    max_queue_size 10 10 10 10 10 10 10 10 10 10 ... 10 10 10 10 10 10 10 10 10 10
    num_train_examples 4800 4800 4800 4800 4800 4800 4800 4800 4800 4800 ... 4800 4800 4800 4800 4800 4800 4800 4800 4800 4800
    num_val_examples 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 ... 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200
    opt_amsgrad False False False False False False False False False False ... False False False False False False False False False False
    opt_beta_1 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 ... 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9
    opt_beta_2 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 ... 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999
    opt_decay 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
    opt_epsilon 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 ... 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001
    opt_learning_rate 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 ... 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001
    opt_name Adam Adam Adam Adam Adam Adam Adam Adam Adam Adam ... Adam Adam Adam Adam Adam Adam Adam Adam Adam Adam
    sample_weight None None None None None None None None None None ... None None None None None None None None None None
    shuffle True True True True True True True True True True ... True True True True True True True True True True
    steps_per_epoch 369 960 436 208 369 369 1200 266 184 320 ... 533 1200 266 480 1200 240 342 960 266 342
    use_multiprocessing False False False False False False False False False False ... False False False False False False False False False False
    validation_batch_size None None None None None None None None None None ... None None None None None None None None None None
    validation_freq 1 1 1 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1 1 1
    validation_split 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
    validation_steps 92 240 109 52 92 92 300 66 46 80 ... 133 300 66 120 300 60 85 240 66 85
    workers 1 1 1 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1 1 1

    35 rows × 38 columns

  4. Now choose the "most successful" runs. I don't just choose a high AUC, but especially a high Recall as well: 
    cond1 = run1_2_combimetrics.loc['val_auc'] > 0.95
cond2 = run1_2_combimetrics.loc['val_recall0.5'] > 0.85
cond3 = run1_2_combimetrics.loc['val_precision0.5'] > 0.85

with pd.option_context('display.max_rows', None, 'display.max_columns', None):  # more options can be specified also
    display(run1_2_combiparams.loc[:, cond1 & cond2 & cond3])
    display(run1_2_combimetrics.loc[:, cond1 & cond2 & cond3])
    (9051e32b87d84f3485b980067addec30, 61ff87bdb89b4e2ba64f8dacc774992d) (93b168c0ff7942c8a908a94129daf973, f243b3b742de4dbcb7ccfbd4244706f8) (a5b8551144ff46e697a39cd1551e1475, 98cf8cdef9c54b5286e277e75e2ab8c1) (00f2635d9fa2463c9a066722163405be, d0a8e1748b194f3290d471b6b44f19f8) (5604d43c1ece461b8e6eaa0dfb65d6dc, 3612536a77f34f22bc83d1d809140aa6) (7cafab027cdd4fc9bf20a43e989df510, 16dff15d935f45e2a836b1f41b07b4e3) (0e328920e86049928202db95e8cfb7be, bf9d2725eb16462d9a101f0a077ce2b5) (1c954fbc02b747bc813c587ac703c74a, ba49a80c2616407a8f1fe1fd12096fe0) (3cbd945b62ec4634839372e403f6f377, 458b36a70db843719d202a8eda448f17)
    Run ID:
    Run Name: {nan} {nan} {nan} {nan} {nan} {nan} {nan} {nan} {nan}
    Start Time: {2021-08-08 22:10:33, 2021-08-08 21:29:38} {2021-08-08 21:15:24, 2021-08-08 21:01:49} {2021-08-08 19:41:51, 2021-08-08 19:14:27} {2021-08-08 19:07:59, 2021-08-08 19:01:30} {2021-08-08 12:21:05, 2021-08-08 13:03:41} {2021-08-08 10:37:39, 2021-08-08 10:29:07} {2021-08-08 07:47:04, 2021-08-08 08:10:44} {2021-08-08 05:49:55, 2021-08-08 05:42:04} {2021-08-08 04:30:30, 2021-08-08 04:19:31}
    batch_size None None None None None None None None None
    class_weight None None None None None None None None None
    epochs 20 20 20 20 20 20 20 20 20
    hp_batch_size 26 15 20 28 20 10 14 17 9
    hp_epochs 20 20 20 20 20 20 20 20 20
    hp_first_filters 44 23 78 6 128 16 16 16 64
    hp_input_size 16384 16384 16384 16384 16384 8192 16384 16384 4096
    hp_lr_power 1 7 4 1 1 1 5 5 1
    hp_lr_start 0.0136170138242663 0.0305060808685107 0.0584071108418767 0.0553313915596308 0.043549707353273 0.0627676336651573 0.0192390310290551 0.0101590069352232 0.0100697459464075
    hp_n_levels 7 6 4 5 3 5 9 3 5
    hp_pool_size 2 4 4 4 4 4 2 4 4
    hp_scaler standard quant_g standard minmax standard robust robust l2 maxabs
    initial_epoch 0 0 0 0 0 0 0 0 0
    lr schedule [0.01361701382426631, 0.012936163133052994, 0.... [0.030506080868510657, 0.021303534028133002, 0... [0.05840711084187669, 0.04757295682515132, 0.0... [0.05533139155963077, 0.05256482198164923, 0.0... [0.04354970735327304, 0.041372221985609386, 0.... [0.06276763366515732, 0.05962925198189945, 0.0... [0.019239031029055137, 0.014886795466253868, 0... [0.010159006935223234, 0.007860845910406034, 0... [0.010069745946407459, 0.009566258649087086, 0...
    max_queue_size 10 10 10 10 10 10 10 10 10
    num_train_examples 4800 4800 4800 4800 4800 4800 4800 4800 4800
    num_val_examples 1200 1200 1200 1200 1200 1200 1200 1200 1200
    opt_amsgrad False False False False False False False False False
    opt_beta_1 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9
    opt_beta_2 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999
    opt_decay 0 0 0 0 0 0 0 0 0
    opt_epsilon 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001 0.0000001
    opt_learning_rate 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001
    opt_name Adam Adam Adam Adam Adam Adam Adam Adam Adam
    sample_weight None None None None None None None None None
    shuffle True True True True True True True True True
    steps_per_epoch 184 320 240 171 240 480 342 282 533
    use_multiprocessing False False False False False False False False False
    validation_batch_size None None None None None None None None None
    validation_freq 1 1 1 1 1 1 1 1 1
    validation_split 0 0 0 0 0 0 0 0 0
    validation_steps 46 80 60 42 60 120 85 70 133
    workers 1 1 1 1 1 1 1 1 1
    (9051e32b87d84f3485b980067addec30, 61ff87bdb89b4e2ba64f8dacc774992d) (93b168c0ff7942c8a908a94129daf973, f243b3b742de4dbcb7ccfbd4244706f8) (a5b8551144ff46e697a39cd1551e1475, 98cf8cdef9c54b5286e277e75e2ab8c1) (00f2635d9fa2463c9a066722163405be, d0a8e1748b194f3290d471b6b44f19f8) (5604d43c1ece461b8e6eaa0dfb65d6dc, 3612536a77f34f22bc83d1d809140aa6) (7cafab027cdd4fc9bf20a43e989df510, 16dff15d935f45e2a836b1f41b07b4e3) (0e328920e86049928202db95e8cfb7be, bf9d2725eb16462d9a101f0a077ce2b5) (1c954fbc02b747bc813c587ac703c74a, ba49a80c2616407a8f1fe1fd12096fe0) (3cbd945b62ec4634839372e403f6f377, 458b36a70db843719d202a8eda448f17)
    Run ID:
    accuracy 9.710000e-01 9.600000e-01 9.725000e-01 9.735000e-01 9.665000e-01 9.705000e-01 9.640000e-01 9.615000e-01 9.705000e-01
    auc 9.810000e-01 9.730000e-01 9.820000e-01 9.840000e-01 9.720000e-01 9.810000e-01 9.730000e-01 9.625000e-01 9.800000e-01
    f1 2.640000e-01 2.640000e-01 2.640000e-01 2.650000e-01 2.640000e-01 2.620000e-01 2.650000e-01 2.635000e-01 2.630000e-01
    fn0.1 3.682675e+05 4.621365e+05 3.568190e+05 3.122355e+05 5.600420e+05 1.845900e+05 5.070725e+05 7.945210e+05 1.017630e+05
    fn0.3 7.227145e+05 9.977800e+05 6.926625e+05 6.512465e+05 9.973125e+05 3.611175e+05 1.124122e+06 1.259410e+06 1.843830e+05
    fn0.5 1.136268e+06 1.521807e+06 1.071660e+06 1.004638e+06 1.420861e+06 5.911320e+05 1.642768e+06 1.763400e+06 2.860370e+05
    fn0.7 1.738792e+06 2.319607e+06 1.661646e+06 1.556141e+06 2.051528e+06 8.906325e+05 2.305407e+06 2.409087e+06 4.321115e+05
    fn0.9 2.890862e+06 4.209348e+06 2.778182e+06 2.663333e+06 3.247781e+06 1.448880e+06 3.677228e+06 3.719575e+06 7.203375e+05
    fp0.1 3.958967e+06 5.572852e+06 3.626632e+06 3.504270e+06 4.532016e+06 1.989762e+06 5.707579e+06 4.813038e+06 9.582675e+05
    fp0.3 1.991625e+06 2.699346e+06 1.887159e+06 1.789370e+06 2.098466e+06 1.039760e+06 2.281552e+06 2.328430e+06 5.024380e+05
    fp0.5 1.120762e+06 1.617764e+06 1.093034e+06 1.072950e+06 1.197675e+06 5.652505e+05 1.199354e+06 1.254754e+06 2.843205e+05
    fp0.7 5.437065e+05 8.631090e+05 5.284415e+05 5.297935e+05 5.737150e+05 2.738040e+05 5.788060e+05 6.135885e+05 1.421470e+05
    fp0.9 1.503025e+05 2.542905e+05 1.457025e+05 1.390835e+05 1.532515e+05 7.276600e+04 1.507885e+05 1.525270e+05 3.853750e+04
    loss 1.950000e-01 2.620000e-01 1.850000e-01 1.710000e-01 2.485000e-01 1.955000e-01 2.645000e-01 3.070000e-01 2.005000e-01
    lr 6.809000e-04 2.383000e-11 3.650000e-07 3.000000e-03 2.000000e-03 3.000000e-03 6.012000e-09 3.175000e-09 5.035000e-04
    precision0.1 7.455000e-01 6.740000e-01 7.620000e-01 7.690000e-01 7.160000e-01 7.430000e-01 6.675000e-01 6.980000e-01 7.500000e-01
    precision0.3 8.490000e-01 8.030000e-01 8.565000e-01 8.640000e-01 8.395000e-01 8.425000e-01 8.260000e-01 8.205000e-01 8.470000e-01
    precision0.5 9.060000e-01 8.660000e-01 9.090000e-01 9.110000e-01 8.980000e-01 9.045000e-01 8.960000e-01 8.900000e-01 9.040000e-01
    precision0.7 9.495000e-01 9.180000e-01 9.510000e-01 9.515000e-01 9.455000e-01 9.485000e-01 9.435000e-01 9.395000e-01 9.470000e-01
    precision0.9 9.835000e-01 9.680000e-01 9.840000e-01 9.850000e-01 9.825000e-01 9.840000e-01 9.825000e-01 9.815000e-01 9.830000e-01
    recall0.1 9.690000e-01 9.615000e-01 9.700000e-01 9.740000e-01 9.530000e-01 9.690000e-01 9.575000e-01 9.335000e-01 9.660000e-01
    recall0.3 9.395000e-01 9.165000e-01 9.420000e-01 9.455000e-01 9.170000e-01 9.390000e-01 9.060000e-01 8.940000e-01 9.380000e-01
    recall0.5 9.050000e-01 8.730000e-01 9.105000e-01 9.160000e-01 8.815000e-01 9.005000e-01 8.625000e-01 8.520000e-01 9.040000e-01
    recall0.7 8.545000e-01 8.065000e-01 8.610000e-01 8.705000e-01 8.285000e-01 8.500000e-01 8.070000e-01 7.980000e-01 8.545000e-01
    recall0.9 7.580000e-01 6.485000e-01 7.675000e-01 7.775000e-01 7.285000e-01 7.560000e-01 6.925000e-01 6.880000e-01 7.575000e-01
    tn0.1 6.248249e+07 6.110056e+07 6.304679e+07 6.295906e+07 6.214143e+07 3.139506e+07 6.077990e+07 6.181710e+07 1.572065e+07
    tn0.3 6.444982e+07 6.397410e+07 6.478628e+07 6.467399e+07 6.457496e+07 3.234507e+07 6.420593e+07 6.430173e+07 1.617648e+07
    tn0.5 6.532068e+07 6.505565e+07 6.558039e+07 6.539041e+07 6.547576e+07 3.281958e+07 6.528812e+07 6.537538e+07 1.639460e+07
    tn0.7 6.589775e+07 6.581032e+07 6.614497e+07 6.593356e+07 6.609972e+07 3.311101e+07 6.590870e+07 6.601654e+07 1.653677e+07
    tn0.9 6.629115e+07 6.641916e+07 6.652773e+07 6.632426e+07 6.652018e+07 3.331206e+07 6.633670e+07 6.647764e+07 1.664038e+07
    tp0.1 1.157134e+07 1.150763e+07 1.161295e+07 1.167101e+07 1.140972e+07 5.752186e+06 1.145204e+07 1.112024e+07 2.867828e+06
    tp0.3 1.121689e+07 1.097199e+07 1.127710e+07 1.133200e+07 1.097245e+07 5.575658e+06 1.083499e+07 1.065535e+07 2.785208e+06
    tp0.5 1.080334e+07 1.044796e+07 1.089811e+07 1.097861e+07 1.054891e+07 5.345644e+06 1.031634e+07 1.015136e+07 2.683554e+06
    tp0.7 1.020082e+07 9.650160e+06 1.030812e+07 1.042710e+07 9.918238e+06 5.046144e+06 9.653704e+06 9.505670e+06 2.537480e+06
    tp0.9 9.048746e+06 7.760420e+06 9.191586e+06 9.319910e+06 8.721986e+06 4.487896e+06 8.281883e+06 8.195182e+06 2.249254e+06
    val_accuracy 9.705000e-01 9.580000e-01 9.715000e-01 9.730000e-01 9.655000e-01 9.650000e-01 9.620000e-01 9.540000e-01 9.655000e-01
    val_auc 9.810000e-01 9.760000e-01 9.840000e-01 9.870000e-01 9.745000e-01 9.780000e-01 9.760000e-01 9.620000e-01 9.720000e-01
    val_f1 2.765000e-01 2.770000e-01 2.770000e-01 2.765000e-01 2.770000e-01 2.770000e-01 2.775000e-01 2.775000e-01 2.810000e-01
    val_fn0.1 9.953900e+04 1.018855e+05 8.291750e+04 6.491900e+04 1.300915e+05 5.183400e+04 1.083685e+05 1.989905e+05 4.027450e+04
    val_fn0.3 2.065230e+05 2.225190e+05 1.673150e+05 1.418270e+05 2.487260e+05 1.058955e+05 2.645855e+05 3.249370e+05 6.900850e+04
    val_fn0.5 3.215935e+05 3.389065e+05 2.659300e+05 2.205875e+05 3.629365e+05 1.735790e+05 3.913830e+05 4.524360e+05 1.028930e+05
    val_fn0.7 4.900970e+05 5.129575e+05 4.164020e+05 3.444560e+05 5.346575e+05 2.454170e+05 5.597010e+05 6.320105e+05 1.485405e+05
    val_fn0.9 8.184030e+05 9.663815e+05 7.104005e+05 6.000115e+05 8.757840e+05 3.997180e+05 9.522080e+05 9.742990e+05 2.333430e+05
    val_fp0.1 9.453770e+05 1.601886e+06 9.806695e+05 1.017959e+06 1.234908e+06 6.305910e+05 1.691592e+06 1.263437e+06 2.185595e+05
    val_fp0.3 4.490195e+05 8.010390e+05 5.028105e+05 4.986330e+05 5.470645e+05 3.102420e+05 6.715845e+05 7.084290e+05 1.185620e+05
    val_fp0.5 2.523625e+05 4.911005e+05 2.909235e+05 2.976760e+05 3.176580e+05 1.722175e+05 3.474940e+05 4.435775e+05 6.640450e+04
    val_fp0.7 1.222830e+05 2.638220e+05 1.435500e+05 1.506035e+05 1.579470e+05 9.873550e+04 1.668495e+05 2.509655e+05 3.329950e+04
    val_fp0.9 3.509800e+04 7.686200e+04 4.052800e+04 4.137150e+04 4.698050e+04 3.210800e+04 4.534800e+04 9.132200e+04 9.003000e+03
    val_loss 1.985000e-01 2.665000e-01 1.855000e-01 1.735000e-01 2.515000e-01 2.360000e-01 2.725000e-01 3.290000e-01 2.440000e-01
    val_precision0.1 7.635000e-01 6.570000e-01 7.585000e-01 7.495000e-01 7.105000e-01 7.095000e-01 6.420000e-01 6.995000e-01 7.770000e-01
    val_precision0.3 8.675000e-01 7.860000e-01 8.565000e-01 8.560000e-01 8.425000e-01 8.270000e-01 8.110000e-01 7.990000e-01 8.610000e-01
    val_precision0.5 9.180000e-01 8.520000e-01 9.090000e-01 9.065000e-01 8.985000e-01 8.910000e-01 8.880000e-01 8.585000e-01 9.135000e-01
    val_precision0.7 9.560000e-01 9.095000e-01 9.505000e-01 9.480000e-01 9.435000e-01 9.315000e-01 9.395000e-01 9.090000e-01 9.515000e-01
    val_precision0.9 9.850000e-01 9.660000e-01 9.840000e-01 9.835000e-01 9.800000e-01 9.735000e-01 9.795000e-01 9.595000e-01 9.845000e-01
    val_recall0.1 9.685000e-01 9.680000e-01 9.740000e-01 9.790000e-01 9.590000e-01 9.670000e-01 9.655000e-01 9.365000e-01 9.500000e-01
    val_recall0.3 9.345000e-01 9.300000e-01 9.470000e-01 9.540000e-01 9.215000e-01 9.330000e-01 9.160000e-01 8.965000e-01 9.140000e-01
    val_recall0.5 8.975000e-01 8.930000e-01 9.160000e-01 9.290000e-01 8.850000e-01 8.905000e-01 8.750000e-01 8.560000e-01 8.720000e-01
    val_recall0.7 8.445000e-01 8.380000e-01 8.685000e-01 8.890000e-01 8.310000e-01 8.450000e-01 8.215000e-01 7.985000e-01 8.145000e-01
    val_recall0.9 7.400000e-01 6.945000e-01 7.750000e-01 8.060000e-01 7.235000e-01 7.475000e-01 6.965000e-01 6.890000e-01 7.090000e-01
    val_tn0.1 1.550422e+07 1.489606e+07 1.551728e+07 1.515836e+07 1.526304e+07 7.617204e+06 1.466611e+07 1.509811e+07 3.882080e+06
    val_tn0.3 1.600058e+07 1.569691e+07 1.599514e+07 1.567769e+07 1.595088e+07 7.937553e+06 1.568612e+07 1.565312e+07 3.982077e+06
    val_tn0.5 1.619724e+07 1.600685e+07 1.620702e+07 1.587864e+07 1.618029e+07 8.075578e+06 1.601021e+07 1.591797e+07 4.034234e+06
    val_tn0.7 1.632732e+07 1.623412e+07 1.635440e+07 1.602572e+07 1.634000e+07 8.149060e+06 1.619086e+07 1.611058e+07 4.067340e+06
    val_tn0.9 1.641450e+07 1.642108e+07 1.645742e+07 1.613495e+07 1.645097e+07 8.215687e+06 1.631236e+07 1.627022e+07 4.091636e+06
    val_tp0.1 3.046124e+06 3.060968e+06 3.079936e+06 3.026346e+06 3.032762e+06 1.530771e+06 3.030885e+06 2.936426e+06 7.619985e+05
    val_tp0.3 2.939140e+06 2.940335e+06 2.995539e+06 2.949438e+06 2.914128e+06 1.476710e+06 2.874668e+06 2.810479e+06 7.332645e+05
    val_tp0.5 2.824070e+06 2.823948e+06 2.896924e+06 2.870677e+06 2.799918e+06 1.409026e+06 2.747870e+06 2.682980e+06 6.993800e+05
    val_tp0.7 2.655566e+06 2.649896e+06 2.746452e+06 2.746808e+06 2.628196e+06 1.337188e+06 2.579552e+06 2.503406e+06 6.537325e+05
    val_tp0.9 2.327260e+06 2.196472e+06 2.452454e+06 2.491253e+06 2.287070e+06 1.182887e+06 2.187046e+06 2.161117e+06 5.689300e+05

    This looks promising with a wide variety of approaches leading to success. Sadly, only run 00f2635d9fa2463c9a066722163405be had the model saved out for immediate testing. The other models have to be trained again with the same settings.

    Lets look at some plots of successful runs as well: High validation AUC: run1-2_highvalauc.png High validation AUC, high validation precision: run1-2_highvalauc-highvalprecision.png High validation AUC, high validation recall: run1-2_highvalauc-highvalrecall.png High validation AUC, high validation recall, high validation precision: run1-2_highvalauc-highvalrecall-highvalprecision.png

  5. Now lets look at some unsuccessful runs and some interesting runs: High training AUC, low validation AUC (these models overfitted). Here a clear predictor seems to be lr_power=1, a small batch size, and a small pool size. run1-2_highauc-lowvalauc.png Low validation AUC, low validation recall: run1-2_lowvalrecall-lowvalauc.png Low validation precision, but still reasonably high validation AUC: run1-2_lowvalprecision-highvalauc.png Low validation recall, but still reasonably high validation AUC: run1-2_lowvalrecall-highvalauc.png
  6. Lets inspect input sizes. Their might be a slight preference for 2**14 length of traces, but it could also be negligible. Input size of 4096 time steps run1-2_input-size_4096.png Input size of 8192 time steps run1-2_input-size_8192.png Input size of 16384 time steps run1-2_input-size_16384.png
  7. Next lets inspect a complicated hyperparameter, which adjusts the pool size of the encoder, as well as the strides and the kernel size of the decoder. It is connected to the input size and nlevels in model building, as well. The following condition has to hold: input_size >= 2 * pool_size**n_levels. Because of that, a poolsize of 8 is only possible with smaller nlevels. Still, a higher poolsize seems to be useful in this project. pool size, kernel size and strides of 2: run1-2_poolsize-strides-kernelsize_2.png pool size, kernel size and strides of 4: run1-2_poolsize-strides-kernelsize_4.png pool size, kernel size and strides of 8: run1-2_poolsize-strides-kernelsize_8.png
  8. Another complicated hyperparameter is nlevels. It refers to the "depth" of the unet. How many times will the input trace be pooled and skip connections made? This parameter strongly influences the amount of total model parameters and smaller values make the model much more portable and "simpler", which could be viewed as a positive thing itself. A larger value raises the learning capacity of the network, but also raises the possibility of overfitting. Number of unet levels 1 to 3 run1-2_n-levels_1-3.png Number of unet levels 4 to 6 run1-2_n-levels_4-6.png Number of unet levels 7 to 9 run1-2_n-levels_7-9.png
  9. Lets take a look at the different first filters (starting from this value, the number of filters was doubled for each unet level till a maximum of 512 filters was reached). More filters mean more capacity for learning, but also more risk of overfitting and model size. Here, no real trend is visible, so a lower number of filters should be sufficient. from 6 to 45 first filters run1-2_first-filters_6-45.png from 57 to 77 first filters run1-2_first-filters_57-77.png from 100 to 128 first filters run1-2_first-filters_100-128.png
  10. Now lets take a look at the starting learning rates and the learning rate power, which together are used to construct a learning rate schedule. For the learning rate starting value, the lower category seemed to perform better than the upper category. The power of 1 is simpler than the values between 4 and 7. Start learning rate between 0.0065 and 0.035: run1-2_lr-start_0.0065-0.035.png Start learning rate between 0.045 and 0.065: run1-2_lr-start_0.045-0.065.png Start learning rate between 0.065 and 0.1: run1-2_lr-start_0.065-0.1.png Power of learning rate equation = 1, meaning a linear decay: run1-2_lr-power_1.png Power of learning rate equation = 4 to 7, meaning a polynomial decay: run1-2_lr-power_2-7.png
  11. The next parameter is the batch size, which is directly coupled to the steps per epoch of the training (since I chose the step size to equal the total number of examples divided by the batch size). A larger batch size pools more training examples for decisions. We see that batch size >7 seem to be benefitial. Batch size of 2 to 8, with steps per epoch between 600 and 2400 run1-2_batch-size_2-8_higher-steps-per-epoch.png Batch size of 8 to 20, with steps per epoch between 165 and 550 run1-2_batch-size_8-20_lower-steps-per-epoch.png
  12. Lastly, lets take a look at the different scalers. I used a scaler from the beginning, because in learning algorithms this guarantees some numerical stability. As a naive approach I chose one - the Min-Max scaler. After evaluating the first training approaches I noticed that there might be problems with this scaler, if the trace has no artifacts in it, see this example plot, where I plotted a variety of scalers from sklearn on some simulated data: scalers_simulations.png From these I chose Standard, Robust, Max-Abs, Quantile (Gaussian), Min-Max, L1 and L2, because they looked most promising. Now, let's load some of the experimental data and see how these scalers look on this data. 
    path_tb_pex5_egfp = '/beegfs/ye53nis/data/Pablo_structured_experiment'
length_delimiter = 2**13  # for U-Net
bin_for_correlation = 1e6
ptu_1ms, _ = ptu.import_from_ptu(
      path=path_tb_pex5_egfp,
      file_delimiter=2,
      photon_count_bin=1e6,
      verbose=True)
ptu_1ms = ptu_1ms

    1 of 2: /beegfs/ye53nis/data/Pablo_structured_experiment/DiO LUV 10uM in 20 nM AF48822_T273s_1.ptu
2 of 2: /beegfs/ye53nis/data/Pablo_structured_experiment/DiO LUV 10uM in 20 nM AF48821_T260s_1.ptu

    scalers = {'Unscaled data': None,
           'Data after standard scaling (z-score)': 'standard',
           'Data after robust scaling': 'robust',
           'Data after max-abs scaling': 'maxabs',
           'Data after quantile transformation (gaussian pdf)': 'quant_g',
           'Data after min-max scaling': 'minmax',
           'Data after sample-wise L1 normalization (taxicab, LASSO)': 'l1',
           'Data after sample-wise L2 normalization (Euclidian)': 'l2'}
plt.figure(figsize=(16,14), facecolor='white')
for i, (text, s) in enumerate(scalers.items()):
    plt.subplot(3, 3, i+1, title=text)
    if s is None:
        plt.plot(ptu_1ms, alpha=0.75)
    else:
        print(np.array(ptu_1ms.iloc[:, 0]))
        # We have to circumvent the following error because we want to dropna() for scaling:
        # /home/ye53nis/.conda/envs/tf/lib/python3.9/site-packages/numpy/core/_asarray.py:136:
        # VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which
        # is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes)
        # is deprecated. If you meant to do this, you must specify 'dtype=object' when creating
        # the ndarray: return array(a, dtype, copy=False, order=order, subok=True)
        # And the sklearn error:
        # ValueError: Expected 2D array, got 1D array instead:
        # array=[486. 455. 622. ... 580. 522. 682.].
        # Reshape your data either using array.reshape(-1, 1) if your data has a single feature
        # or array.reshape(1, -1) if it contains a single sample.
        tscaled = [ppd.scale_trace(np.array(ptu_1ms.iloc[:, j].dropna(), dtype=object).reshape(-1, 1),
                                   scaler=s) for j in range(2)]
        [plt.plot(tscaled[k], alpha=0.75) for k in range(2)]
plt.tight_layout()

    with reshape(-1, 1) to say that data has a single feature scalers_experimental.png . We already see that the quantile transformation might not be so useful here. In the simulated data with slower cluster speeds it made the peaks clearly distinct, here this is not the case.

    using reshape(1, -1) to say that data contains a single sample does not work e.g. each timestep normalized to 0…

    First, let's look at L1 and L2. L1 was only chosen once, L2 got chosen more often. In runs with the highest val_auc, this norm especially seems to lead to a high recall (which is a problem with minmax). But we also see that depending on the other parameters there is a wide variation of results, including very bad ones run1-2_scaler_l1.png run1-2_scaler_l2.png

    Second, let's look at Max-Abs (from 0 to the maximum value) and Min-Max (from the minimum to the maximum value). Both are conceptually somewhat similar, although Max-Abs keeps the distance from 0 to the minimal value, and Min-Max does not. We see that both lead to good results, with some variation. In Max-Abs, the spread seems to be wider. run1-2_scaler_maxabs.png run1-2_scaler_minmax.png

    Third, Quantile Transformation (Gaussian distribution). We see that this transformation suffers from bad valprecision values. Higher recall values can be reached with other transformations. Thus, I think I can neglect this transformation in the future. run1-2_scaler_quant-g.png

    Fourth, the Robust Scaler. It robustly (haha) achieves good precision and recall values run1-2_scaler_robust.png

    Fifth, the Standard Scaler. It achieves some of the best results, even though there is some variation, both in precision and recall. run1-2_scaler_standard.png

2.1.5 Look at training logging of training run 00f2635d9fa2463c9a066722163405be

This run is one of the few very good runs where the model got saved out.

  1. Let's look at some prediction plots after 1 epoch: plot0.png after 5 epochs: plot4.png after 10 epochs: plot9.png after 15 epochs: plot14.png after 20 epochs: plot19.png
  2. Now let's print out the model architecture. -n prints out line numbers. 
    cat -n data/mlruns/8/00f2635d9fa2463c9a066722163405be/artifacts/model_summary.txt

    |  1 | unet_depth5                                                                                        |
|  2 | __________________________________________________________________________________________________ |
|  3 | Layer (type)                    Output Shape         Param #     Connected to                      |
|  4 | ================================================================================================== |
|  5 | input_6 (InputLayer)            [(None, 16384, 1)]   0                                             |
|  6 | __________________________________________________________________________________________________ |
|  7 | encode0 (Sequential)            (None, 16384, 6)     186         input_6[0][0]                     |
|  8 | __________________________________________________________________________________________________ |
|  9 | mp_encode0 (MaxPooling1D)       (None, 4096, 6)      0           encode0[0][0]                     |
| 10 | __________________________________________________________________________________________________ |
| 11 | encode1 (Sequential)            (None, 4096, 12)     768         mp_encode0[0][0]                  |
| 12 | __________________________________________________________________________________________________ |
| 13 | mp_encode1 (MaxPooling1D)       (None, 1024, 12)     0           encode1[0][0]                     |
| 14 | __________________________________________________________________________________________________ |
| 15 | encode2 (Sequential)            (None, 1024, 24)     2832        mp_encode1[0][0]                  |
| 16 | __________________________________________________________________________________________________ |
| 17 | mp_encode2 (MaxPooling1D)       (None, 256, 24)      0           encode2[0][0]                     |
| 18 | __________________________________________________________________________________________________ |
| 19 | encode3 (Sequential)            (None, 256, 48)      10848       mp_encode2[0][0]                  |
| 20 | __________________________________________________________________________________________________ |
| 21 | mp_encode3 (MaxPooling1D)       (None, 64, 48)       0           encode3[0][0]                     |
| 22 | __________________________________________________________________________________________________ |
| 23 | encode4 (Sequential)            (None, 64, 96)       42432       mp_encode3[0][0]                  |
| 24 | __________________________________________________________________________________________________ |
| 25 | mp_encode4 (MaxPooling1D)       (None, 16, 96)       0           encode4[0][0]                     |
| 26 | __________________________________________________________________________________________________ |
| 27 | two_conv_center (Sequential)    (None, 16, 192)      167808      mp_encode4[0][0]                  |
| 28 | __________________________________________________________________________________________________ |
| 29 | conv_transpose_decoder4 (Sequen (None, 64, 192)      148416      two_conv_center[0][0]             |
| 30 | __________________________________________________________________________________________________ |
| 31 | decoder4 (Concatenate)          (None, 64, 288)      0           encode4[0][0]                     |
| 32 | conv_transpose_decoder4[0][0]                                                                      |
| 33 | __________________________________________________________________________________________________ |
| 34 | two_conv_decoder4 (Sequential)  (None, 64, 192)      278400      decoder4[0][0]                    |
| 35 | __________________________________________________________________________________________________ |
| 36 | conv_transpose_decoder3 (Sequen (None, 256, 96)      74208       two_conv_decoder4[0][0]           |
| 37 | __________________________________________________________________________________________________ |
| 38 | decoder3 (Concatenate)          (None, 256, 144)     0           encode3[0][0]                     |
| 39 | conv_transpose_decoder3[0][0]                                                                      |
| 40 | __________________________________________________________________________________________________ |
| 41 | two_conv_decoder3 (Sequential)  (None, 256, 96)      70080       decoder3[0][0]                    |
| 42 | __________________________________________________________________________________________________ |
| 43 | conv_transpose_decoder2 (Sequen (None, 1024, 48)     18672       two_conv_decoder3[0][0]           |
| 44 | __________________________________________________________________________________________________ |
| 45 | decoder2 (Concatenate)          (None, 1024, 72)     0           encode2[0][0]                     |
| 46 | conv_transpose_decoder2[0][0]                                                                      |
| 47 | __________________________________________________________________________________________________ |
| 48 | two_conv_decoder2 (Sequential)  (None, 1024, 48)     17760       decoder2[0][0]                    |
| 49 | __________________________________________________________________________________________________ |
| 50 | conv_transpose_decoder1 (Sequen (None, 4096, 24)     4728        two_conv_decoder2[0][0]           |
| 51 | __________________________________________________________________________________________________ |
| 52 | decoder1 (Concatenate)          (None, 4096, 36)     0           encode1[0][0]                     |
| 53 | conv_transpose_decoder1[0][0]                                                                      |
| 54 | __________________________________________________________________________________________________ |
| 55 | two_conv_decoder1 (Sequential)  (None, 4096, 24)     4560        decoder1[0][0]                    |
| 56 | __________________________________________________________________________________________________ |
| 57 | conv_transpose_decoder0 (Sequen (None, 16384, 12)    1212        two_conv_decoder1[0][0]           |
| 58 | __________________________________________________________________________________________________ |
| 59 | decoder0 (Concatenate)          (None, 16384, 18)    0           encode0[0][0]                     |
| 60 | conv_transpose_decoder0[0][0]                                                                      |
| 61 | __________________________________________________________________________________________________ |
| 62 | two_conv_decoder0 (Sequential)  (None, 16384, 12)    1200        decoder0[0][0]                    |
| 63 | __________________________________________________________________________________________________ |
| 64 | conv1d_89 (Conv1D)              (None, 16384, 1)     13          two_conv_decoder0[0][0]           |
| 65 | ================================================================================================== |
| 66 | Total params: 844,123                                                                              |
| 67 | Trainable params: 840,379                                                                          |
| 68 | Non-trainable params: 3,744                                                                        |
| 69 | __________________________________________________________________________________________________ |
  3. Print out mlflow Model parameters 
    cat -n data/mlruns/8/00f2635d9fa2463c9a066722163405be/artifacts/model/MLmodel

    |  1 | artifact_path: model                           |
|  2 | flavors:                                       |
|  3 | keras:                                         |
|  4 | data: data                                     |
|  5 | keras_module: tensorflow.keras                 |
|  6 | keras_version: 2.5.0                           |
|  7 | save_format: tf                                |
|  8 | python_function:                               |
|  9 | data: data                                     |
| 10 | env: conda.yaml                                |
| 11 | loader_module: mlflow.keras                    |
| 12 | python_version: 3.9.6                          |
| 13 | run_id: 00f2635d9fa2463c9a066722163405be       |
| 14 | utc_time_created: '2021-08-08 17:14:06.771348' |
  4. Print out training data parameters. One learning: The f1-metric does not work! 
    echo "time val_recall0.5 epoch"
cat data/mlruns/8/00f2635d9fa2463c9a066722163405be/metrics/val_recall0.5
echo ""
echo "time val_precision0.5 epoch"
cat data/mlruns/8/00f2635d9fa2463c9a066722163405be/metrics/val_precision0.5
echo ""
echo "time val_auc epoch"
cat data/mlruns/8/00f2635d9fa2463c9a066722163405be/metrics/val_auc
echo ""
echo "time val_f1 epoch"
cat data/mlruns/8/00f2635d9fa2463c9a066722163405be/metrics/val_f1
    time val\recall0.5 epoch
    1628442521951 1.0 0
    1628442538552 1.0 1
    1628442555029 0.9884297847747803 2
    1628442571921 0.9939945340156555 3
    1628442588629 0.9871219396591187 4
    1628442605537 0.9595196843147278 5
    1628442622450 0.9733913540840149 6
    1628442642363 0.9914226531982422 7
    1628442659452 0.9950642585754395 8
    1628442676492 0.7978640794754028 9
    1628442693576 0.9715087413787842 10
    1628442710632 0.9640132188796997 11
    1628442727806 0.9724437594413757 12
    1628442744594 0.9310197234153748 13
    1628442761707 0.8998530507087708 14
    1628442778890 0.9342276453971863 15
    1628442795829 0.9401260018348694 16
    1628442812832 0.9072213768959045 17
    1628442829592 0.8626136183738708 18
    1628442846600 0.9356982707977295 19
         
    time val\precision0.5 epoch
    1628442521951 0.161118745803833 0
    1628442538552 0.16019564867019653 1
    1628442555029 0.17915846407413483 2
    1628442571921 0.243782639503479 3
    1628442588629 0.2357945740222931 4
    1628442605537 0.24214163422584534 5
    1628442622450 0.5458978414535522 6
    1628442642363 0.4380616247653961 7
    1628442659452 0.2929491698741913 8
    1628442676492 0.8729878067970276 9
    1628442693576 0.7323421835899353 10
    1628442710632 0.5339199304580688 11
    1628442727806 0.7172960042953491 12
    1628442744594 0.8843752145767212 13
    1628442761707 0.9003584980964661 14
    1628442778890 0.8853111267089844 15
    1628442795829 0.8726167678833008 16
    1628442812832 0.9290499091148376 17
    1628442829592 0.9494153261184692 18
    1628442846600 0.8976160287857056 19
         
    time val\auc epoch
    1628442521951 0.5 0
    1628442538552 0.5000137686729431 1
    1628442555029 0.5814657211303711 2
    1628442571921 0.8558493852615356 3
    1628442588629 0.7804751396179199 4
    1628442605537 0.7518471479415894 5
    1628442622450 0.9729548096656799 6
    1628442642363 0.980221688747406 7
    1628442659452 0.9016830325126648 8
    1628442676492 0.9798668026924133 9
    1628442693576 0.9884295463562012 10
    1628442710632 0.9497971534729004 11
    1628442727806 0.9887964129447937 12
    1628442744594 0.9877431392669678 13
    1628442761707 0.9804221987724304 14
    1628442778890 0.9873037338256836 15
    1628442795829 0.9889394640922546 16
    1628442812832 0.9840959310531616 17
    1628442829592 0.9767916202545166 18
    1628442846600 0.9875953793525696 19
         
    time val\f1 epoch
    1628442521951 0.2775232493877411 0
    1628442538552 0.2761525511741638 1
    1628442555029 0.27699539065361023 2
    1628442571921 0.27680331468582153 3
    1628442588629 0.27761173248291016 4
    1628442605537 0.2781853973865509 5
    1628442622450 0.27713820338249207 6
    1628442642363 0.27754780650138855 7
    1628442659452 0.27742522954940796 8
    1628442676492 0.2759542465209961 9
    1628442693576 0.27791914343833923 10
    1628442710632 0.2769533693790436 11
    1628442727806 0.2773088812828064 12
    1628442744594 0.2751254439353943 13
    1628442761707 0.2785097658634186 14
    1628442778890 0.27688315510749817 15
    1628442795829 0.2764797508716583 16
    1628442812832 0.27767083048820496 17
    1628442829592 0.2765066623687744 18
    1628442846600 0.27736371755599976 19
  5. Of course I have plotted out a lot of further info (tensorflow, mlflow, experiment parameters, etc). But I will plot this out in the dedicated training run, not in the hparams run.

2.1.6 Apply model from run 2 to experimental data

%cd /beegfs/ye53nis/drmed-git

/beegfs/ye53nis/drmed-git

from pathlib import Path

import sys
import mlflow
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
import tensorflow as tf
print("tf version: ", tf.version.VERSION)
print("tf.keras version: ", tf.keras.__version__)
print("mlflow version: ", mlflow.version.VERSION)

tf version:  2.5.0
tf.keras version:  2.5.0
mlflow version:  1.19.0

sys.path.append('src/')
from fluotracify.simulations import (
   import_simulation_from_csv as isfc,
   analyze_simulations as ans,
)
from fluotracify.training import build_model as bm, preprocess_data as ppd
from fluotracify.applications import correlate, plots, correction
from fluotracify.imports import ptu_utils as ptu

import importlib
importlib.reload(correction)

logged_model = 'file:///beegfs/ye53nis/drmed-git/data/mlruns/8/00f2635d9fa2463c9a066722163405be/artifacts/model'
metrics_thresholds = [0.1, 0.3, 0.5, 0.7, 0.9]

loaded_model = mlflow.keras.load_model(logged_model, compile=False)
loaded_model.compile(loss=bm.binary_ce_dice_loss(),
                     optimizer=tf.keras.optimizers.Adam(),
                     metrics = bm.unet_metrics(metrics_thresholds))
loaded_model

WARNING:tensorflow:Unable to restore custom metric. Please ensure that the layer implements `get_config` and `from_config` when saving. In addition, please use the `custom_objects` arg when calling `load_model()`.
<tensorflow.python.keras.engine.functional.Functional at 0x2b21c027e8b0>

# The following block has to be executed multiple times with different array lengths
# e.g. 2**14, 2**13, 2**12
# note that: after executing 2**14, then 2**13, there will be an error
# if you then execute 2**12 and then 2**13 again, the error will be gone
test_features = np.zeros((2**14)).reshape(1, -1, 1)
print(test_features.shape)
predictions = loaded_model.predict(test_features, verbose=0).flatten()
predictions

(1, 16384, 1)
array([1.4064237e-01, 1.3650321e-08, 7.4799689e-10, ..., 3.1755112e-10,
       1.8236226e-10, 6.6588524e-10], dtype=float32)

test_features2 = np.zeros((2**13)).reshape(1, -1, 1)
print(test_features2.shape)
predictions = loaded_model.predict(test_features2, verbose=0).flatten()
predictions

(1, 8192, 1)
array([1.4064237e-01, 1.3650321e-08, 7.4799689e-10, ..., 3.1755112e-10,
       1.8236226e-10, 6.6588524e-10], dtype=float32)

path_tb_pex5_egfp = '/beegfs/ye53nis/data/Pablo_structured_experiment'
pred_thresh = [0.1, 0.3, 0.5, 0.7, 0.9]
length_delimiter = 2**13  # for U-Net
bin_for_correlation = 1e6
out = correction.correct_experimental_traces_from_ptu_by_unet_prediction(
  path_list=path_tb_pex5_egfp,
  model=loaded_model,
  pred_thresh=pred_thresh,
  photon_count_bin=bin_for_correlation,
  ntraces=None,
  save_as_csv=True)
out

Loading dataset 1 from path /beegfs/ye53nis/data/Pablo_structured_experiment with bin=1e6. This can take a while...
1 of 2: /beegfs/ye53nis/data/Pablo_structured_experiment/DiO LUV 10uM in 20 nM AF48822_T273s_1.ptu
2 of 2: /beegfs/ye53nis/data/Pablo_structured_experiment/DiO LUV 10uM in 20 nM AF48821_T260s_1.ptu
Processing correlation of unprocessed dataset 1
Processing correlation with correction by prediction of dataset 1
  \(D\) in \(\frac{{\mu m^2}}{{s}}\) \(\tau_{{D}}\) in \(ms\) Trace lengths folderid-tracesused Photon count bin for correlation in \(ns\) FileGUID FileCreatingTime MeasurementSubMode FileComment TTResultStopReason … MeasDescGlobalResolution TTResultNumberOfRecords MeasDescAcquisitionTime TTResultMDescWarningFlags TTResultStopAfter TTResultFormatTTTRRecType TTResultFormatBitsPerRecord UsrPowerDiode HeaderEnd Number of Channels
0 0.898516 12.544077 8192 0-orig 1000000.0 {1FADBD63-E853-4886-82EB-0D4FEB0A2641} (2019, 10, 31, 11, 17, 29, 3, 304, 0) 0   0 … 0.0 6124808 10000 0 10000 16843524 32 1002.253351 <empty Tag> 1
1 1.488758 7.570775 8192 0-orig 1000000.0 {1104778F-509F-40AB-9176-580B3A1CE38D} (2019, 10, 31, 11, 17, 16, 3, 304, 0) 0   0 … 0.0 5881858 10000 0 10000 16843524 32 979.946455 <empty Tag> 1
2 7.747958 1.454713 5053 0-pred-0.1 1000000.0 {1FADBD63-E853-4886-82EB-0D4FEB0A2641} (2019, 10, 31, 11, 17, 29, 3, 304, 0) 0   0 … 0.0 6124808 10000 0 10000 16843524 32 1002.253351 <empty Tag> 1
3 2.388373 4.719135 5926 0-pred-0.1 1000000.0 {1104778F-509F-40AB-9176-580B3A1CE38D} (2019, 10, 31, 11, 17, 16, 3, 304, 0) 0   0 … 0.0 5881858 10000 0 10000 16843524 32 979.946455 <empty Tag> 1
4 5.875199 1.918413 5729 0-pred-0.3 1000000.0 {1FADBD63-E853-4886-82EB-0D4FEB0A2641} (2019, 10, 31, 11, 17, 29, 3, 304, 0) 0   0 … 0.0 6124808 10000 0 10000 16843524 32 1002.253351 <empty Tag> 1
5 2.144527 5.255731 6707 0-pred-0.3 1000000.0 {1104778F-509F-40AB-9176-580B3A1CE38D} (2019, 10, 31, 11, 17, 16, 3, 304, 0) 0   0 … 0.0 5881858 10000 0 10000 16843524 32 979.946455 <empty Tag> 1
6 3.673109 3.068532 6179 0-pred-0.5 1000000.0 {1FADBD63-E853-4886-82EB-0D4FEB0A2641} (2019, 10, 31, 11, 17, 29, 3, 304, 0) 0   0 … 0.0 6124808 10000 0 10000 16843524 32 1002.253351 <empty Tag> 1
7 1.816449 6.204994 7164 0-pred-0.5 1000000.0 {1104778F-509F-40AB-9176-580B3A1CE38D} (2019, 10, 31, 11, 17, 16, 3, 304, 0) 0   0 … 0.0 5881858 10000 0 10000 16843524 32 979.946455 <empty Tag> 1
8 1.857476 6.067942 6576 0-pred-0.7 1000000.0 {1FADBD63-E853-4886-82EB-0D4FEB0A2641} (2019, 10, 31, 11, 17, 29, 3, 304, 0) 0   0 … 0.0 6124808 10000 0 10000 16843524 32 1002.253351 <empty Tag> 1
9 1.45624 7.739836 7446 0-pred-0.7 1000000.0 {1104778F-509F-40AB-9176-580B3A1CE38D} (2019, 10, 31, 11, 17, 16, 3, 304, 0) 0   0 … 0.0 5881858 10000 0 10000 16843524 32 979.946455 <empty Tag> 1
10 0.986952 11.420058 7081 0-pred-0.9 1000000.0 {1FADBD63-E853-4886-82EB-0D4FEB0A2641} (2019, 10, 31, 11, 17, 29, 3, 304, 0) 0   0 … 0.0 6124808 10000 0 10000 16843524 32 1002.253351 <empty Tag> 1
11 1.150654 9.795347 7735 0-pred-0.9 1000000.0 {1104778F-509F-40AB-9176-580B3A1CE38D} (2019, 10, 31, 11, 17, 16, 3, 304, 0) 0   0 … 0.0 5881858 10000 0 10000 16843524 32 979.946455 <empty Tag> 1

12 rows × 94 columns 

path_pex5_exp = ['/beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu', '/beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu']
pred_thresh = [0.1, 0.3, 0.5, 0.7, 0.9]
length_delimiter = 2**13  # for U-Net
bin_for_correlation = 1e5
out = correction.correct_experimental_traces_from_ptu_by_unet_prediction(
  path_list=path_pex5_exp,
  model=loaded_model,
  pred_thresh=pred_thresh,
  photon_count_bin=bin_for_correlation,
  ntraces=400,
  save_as_csv=True)
out

Loading dataset 1 from path /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu with bin=1e6. This can take a while...
1 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48891_T1082s_1.ptu
2 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488129_T1537s_1.ptu
3 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488210_T2505s_1.ptu
4 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488199_T2375s_1.ptu
5 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488171_T2040s_1.ptu
6 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488229_T2733s_1.ptu
7 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48890_T1070s_1.ptu
8 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488113_T1352s_1.ptu
9 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48883_T986s_1.ptu10 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488227_T2709s_1.ptu
11 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488174_T2088s_1.ptu
12 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488102_T1214s_1.ptu
13 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488131_T1561s_1.ptu
14 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48826_T302s_1.ptu
15 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488162_T1932s_1.ptu
16 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488190_T2267s_1.ptu
17 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488189_T2255s_1.ptu
18 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488208_T2481s_1.ptu
19 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48827_T314s_1.ptu
20 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488221_T2637s_1.ptu
21 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488153_T1834s_1.ptu
22 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488157_T1883s_1.ptu
23 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488197_T2351s_1.ptu
24 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48822_T254s_1.ptu
25 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48816_T182s_1.ptu
26 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488179_T2148s_1.ptu
27 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488104_T1238s_1.ptu
28 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48870_T833s_1.ptu
29 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48857_T676s_1.ptu
30 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488125_T1496s_1.ptu
31 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48839_T459s_1.ptu
32 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488135_T1610s_1.ptu
33 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48850_T592s_1.ptu
34 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488159_T1907s_1.ptu
35 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48851_T604s_1.ptu
36 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488158_T1895s_1.ptu
37 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48878_T926s_1.ptu
38 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48838_T447s_1.ptu
39 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48869_T821s_1.ptu
40 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48834_T399s_1.ptu
41 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48843_T506s_1.ptu
42 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48890_T1076s_1.ptu
43 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488184_T2209s_1.ptu
44 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48863_T748s_1.ptu
45 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488103_T1232s_1.ptu
46 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488181_T2159s_1.ptu
47 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48859_T698s_1.ptu
48 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48872_T855s_1.ptu
49 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488150_T1789s_1.ptu
50 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488241_T2877s_1.ptu
51 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488204_T2434s_1.ptu
52 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48877_T918s_1.ptu
53 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488180_T2147s_1.ptu
54 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488183_T2197s_1.ptu
55 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488160_T1908s_1.ptu
56 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488238_T2842s_1.ptu
57 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488165_T1968s_1.ptu
58 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488234_T2793s_1.ptu
59 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48882_T979s_1.ptu
60 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48896_T1148s_1.ptu
61 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488107_T1280s_1.ptu
62 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488138_T1654s_1.ptu
63 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488124_T1484s_1.ptu
64 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488187_T2245s_1.ptu
65 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48874_T879s_1.ptu
66 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488159_T1896s_1.ptu
67 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488177_T2124s_1.ptu
68 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488111_T1328s_1.ptu
69 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48873_T867s_1.ptu
70 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48881_T962s_1.ptu
71 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48853_T628s_1.ptu
72 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48842_T496s_1.ptu
73 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48850_T590s_1.ptu
74 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48899_T1184s_1.ptu
75 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488196_T2339s_1.ptu
76 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488164_T1956s_1.ptu
77 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488217_T2589s_1.ptu
78 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488218_T2601s_1.ptu
79 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488119_T1418s_1.ptu
80 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488206_T2458s_1.ptu
81 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48866_T785s_1.ptu
82 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488103_T1226s_1.ptu
83 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48824_T278s_1.ptu
84 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF4887_T74s_1.ptu
85 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488176_T2099s_1.ptu
86 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48817_T194s_1.ptu
87 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488113_T1346s_1.ptu
88 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48858_T688s_1.ptu
89 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488211_T2517s_1.ptu
90 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48860_T712s_1.ptu
91 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48833_T387s_1.ptu
92 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48858_T686s_1.ptu
93 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48813_T146s_1.ptu
94 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488126_T1502s_1.ptu
95 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48894_T1118s_1.ptu
96 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488110_T1316s_1.ptu
97 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488188_T2243s_1.ptu
98 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488120_T1430s_1.ptu
99 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488219_T2613s_1.ptu
100 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488138_T1646s_1.ptu
101 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488187_T2231s_1.ptu
102 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488146_T1742s_1.ptu
103 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488155_T1859s_1.ptu
104 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488174_T2075s_1.ptu
105 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488158_T1884s_1.ptu
106 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488244_T2914s_1.ptu
107 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF4886_T62s_1.ptu
108 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48885_T1015s_1.ptu
109 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488165_T1979s_1.ptu
110 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48857_T674s_1.ptu
111 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488191_T2279s_1.ptu
112 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488183_T2183s_1.ptu
113 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488141_T1689s_1.ptu
114 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488189_T2269s_1.ptu
115 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48823_T266s_1.ptu
116 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488209_T2493s_1.ptu
117 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488136_T1630s_1.ptu
118 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488185_T2221s_1.ptu
119 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488179_T2135s_1.ptu
120 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48866_T782s_1.ptu
121 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488123_T1472s_1.ptu
122 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488160_T1919s_1.ptu
123 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48856_T664s_1.ptu
124 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48830_T351s_1.ptu
125 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48832_T375s_1.ptu
126 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48859_T700s_1.ptu
127 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488129_T1545s_1.ptu
128 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488143_T1714s_1.ptu
129 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48886_T1022s_1.ptu
130 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488147_T1754s_1.ptu
131 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488193_T2303s_1.ptu
132 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488105_T1250s_1.ptu
133 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488228_T2721s_1.ptu
134 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48891_T1088s_1.ptu
135 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488170_T2040s_1.ptu
136 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48835_T411s_1.ptu
137 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48888_T1046s_1.ptu
138 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488123_T1466s_1.ptu
139 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48876_T906s_1.ptu
140 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48822_T255s_1.ptu
141 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488145_T1730s_1.ptu
142 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488114_T1358s_1.ptu
143 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48824_T279s_1.ptu
144 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48854_T638s_1.ptu
145 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488247_T2949s_1.ptu
146 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48844_T519s_1.ptu
147 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48841_T482s_1.ptu
148 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48828_T327s_1.ptu
149 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48840_T471s_1.ptu
150 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488225_T2685s_1.ptu
151 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48831_T362s_1.ptu
152 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48849_T578s_1.ptu
153 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48861_T722s_1.ptu
154 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48852_T614s_1.ptu
155 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48852_T616s_1.ptu
156 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488149_T1778s_1.ptu
157 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488169_T2028s_1.ptu
158 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF4883_T26s_1.ptu
159 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48881_T966s_1.ptu
160 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48837_T435s_1.ptu
161 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488139_T1666s_1.ptu
162 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488194_T2315s_1.ptu
163 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488205_T2446s_1.ptu
164 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48818_T206s_1.ptu
165 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488128_T1533s_1.ptu
166 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48851_T602s_1.ptu
167 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488116_T1388s_1.ptu
168 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488184_T2196s_1.ptu
169 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488117_T1394s_1.ptu
170 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48869_T818s_1.ptu
171 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48895_T1130s_1.ptu
172 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488173_T2076s_1.ptu
173 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488106_T1262s_1.ptu
174 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488249_T2993s_1.ptu
175 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48889_T1063s_1.ptu
176 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48845_T531s_1.ptu
177 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488117_T1400s_1.ptu
178 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488192_T2291s_1.ptu
179 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488166_T1992s_1.ptu
180 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48896_T1142s_1.ptu
181 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48894_T1124s_1.ptu
182 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488177_T2111s_1.ptu
183 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48862_T736s_1.ptu
184 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488118_T1412s_1.ptu
185 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48892_T1100s_1.ptu
186 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48847_T555s_1.ptu
187 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488223_T2661s_1.ptu
188 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48853_T626s_1.ptu
189 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488136_T1622s_1.ptu
190 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488151_T1810s_1.ptu
191 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488245_T2925s_1.ptu
192 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48879_T942s_1.ptu
193 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488121_T1448s_1.ptu
194 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48868_T809s_1.ptu
195 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488201_T2398s_1.ptu
196 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488140_T1670s_1.ptu
197 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488250_T3006s_1.ptu
198 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF4884_T38s_1.ptu
199 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488127_T1514s_1.ptu
200 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488104_T1244s_1.ptu
201 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488181_T2172s_1.ptu
202 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488115_T1370s_1.ptu
203 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488140_T1678s_1.ptu
204 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48855_T652s_1.ptu
205 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48884_T1003s_1.ptu
206 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48846_T544s_1.ptu
207 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488131_T1569s_1.ptu
208 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488236_T2817s_1.ptu
209 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488178_T2123s_1.ptu
210 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48873_T869s_1.ptu
211 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488164_T1968s_1.ptu
212 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488249_T2973s_1.ptu
213 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488151_T1801s_1.ptu
214 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488235_T2805s_1.ptu
215 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488170_T2028s_1.ptu
216 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488180_T2160s_1.ptu
217 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488135_T1618s_1.ptu
218 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488190_T2281s_1.ptu
219 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48864_T758s_1.ptu
220 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48867_T794s_1.ptu
221 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48875_T891s_1.ptu
222 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48871_T842s_1.ptu
223 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488213_T2541s_1.ptu
224 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488182_T2184s_1.ptu
225 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48841_T484s_1.ptu
226 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488171_T2052s_1.ptu
227 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488153_T1825s_1.ptu
228 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488132_T1581s_1.ptu
229 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488147_T1762s_1.ptu
230 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488110_T1310s_1.ptu
231 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48885_T1010s_1.ptu
232 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48825_T290s_1.ptu
233 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488168_T2004s_1.ptu
234 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48820_T230s_1.ptu
235 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488162_T1944s_1.ptu
236 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488203_T2422s_1.ptu
237 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488152_T1822s_1.ptu
238 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48821_T243s_1.ptu
239 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488167_T2004s_1.ptu
240 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48837_T436s_1.ptu
241 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488166_T1980s_1.ptu
242 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488200_T2386s_1.ptu
243 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488122_T1454s_1.ptu
244 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48843_T508s_1.ptu
245 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488102_T1220s_1.ptu
246 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48865_T773s_1.ptu
247 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488161_T1920s_1.ptu
248 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48823_T267s_1.ptu
249 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48863_T746s_1.ptu
250 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488132_T1573s_1.ptu
251 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48867_T797s_1.ptu
252 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488144_T1726s_1.ptu
253 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488145_T1738s_1.ptu
254 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488169_T2016s_1.ptu
255 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48889_T1058s_1.ptu
256 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48860_T710s_1.ptu
257 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48831_T363s_1.ptu
258 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488215_T2565s_1.ptu
259 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48878_T930s_1.ptu
260 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48865_T770s_1.ptu
261 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48870_T830s_1.ptu
262 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488114_T1364s_1.ptu
263 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488148_T1774s_1.ptu
264 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488224_T2673s_1.ptu
265 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488172_T2052s_1.ptu
266 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488167_T1992s_1.ptu
267 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488100_T1190s_1.ptu
268 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48821_T242s_1.ptu
269 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488240_T2865s_1.ptu
270 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488142_T1701s_1.ptu
271 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488130_T1549s_1.ptu
272 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48844_T520s_1.ptu
273 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48861_T724s_1.ptu
274 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48814_T158s_1.ptu
275 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488163_T1956s_1.ptu
276 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488173_T2064s_1.ptu
277 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF4885_T50s_1.ptu
278 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48849_T580s_1.ptu
279 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48840_T472s_1.ptu
280 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48883_T991s_1.ptu
281 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488247_T2969s_1.ptu
282 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488150_T1798s_1.ptu
283 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48848_T566s_1.ptu
284 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48897_T1160s_1.ptu
285 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488118_T1406s_1.ptu
286 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF4889_T98s_1.ptu
287 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48812_T134s_1.ptu
288 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488222_T2649s_1.ptu
289 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48887_T1039s_1.ptu
290 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48879_T938s_1.ptu
291 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48888_T1051s_1.ptu
292 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48876_T902s_1.ptu
293 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48819_T218s_1.ptu
294 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48893_T1112s_1.ptu
295 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488121_T1442s_1.ptu
296 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488101_T1208s_1.ptu
297 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488109_T1304s_1.ptu
298 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488128_T1526s_1.ptu
299 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488207_T2470s_1.ptu
300 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488246_T2937s_1.ptu
301 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488242_T2890s_1.ptu
302 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48854_T640s_1.ptu
303 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48847_T554s_1.ptu
304 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488108_T1292s_1.ptu
305 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488134_T1597s_1.ptu
306 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF4888_T86s_1.ptu
307 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48872_T857s_1.ptu
308 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488149_T1786s_1.ptu
309 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48825_T291s_1.ptu
310 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488182_T2171s_1.ptu
311 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488156_T1871s_1.ptu
312 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488220_T2624s_1.ptu
313 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488248_T2981s_1.ptu
314 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48897_T1154s_1.ptu
315 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48848_T568s_1.ptu
316 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488175_T2100s_1.ptu
317 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488154_T1847s_1.ptu
318 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488245_T2945s_1.ptu
319 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488139_T1658s_1.ptu
320 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48830_T350s_1.ptu
321 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488175_T2087s_1.ptu
322 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48856_T662s_1.ptu
323 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488230_T2745s_1.ptu
324 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48875_T894s_1.ptu
325 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488127_T1521s_1.ptu
326 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488195_T2327s_1.ptu
327 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48884_T998s_1.ptu
328 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488186_T2219s_1.ptu
329 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48882_T974s_1.ptu
330 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488134_T1606s_1.ptu
331 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488108_T1286s_1.ptu
332 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48827_T315s_1.ptu
333 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488214_T2553s_1.ptu
334 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488163_T1944s_1.ptu
335 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48839_T460s_1.ptu
336 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488186_T2233s_1.ptu
337 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488243_T2902s_1.ptu
338 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488137_T1642s_1.ptu
339 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488156_T1861s_1.ptu
340 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF4882_T14s_1.ptu
341 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488142_T1694s_1.ptu
342 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48811_T123s_1.ptu
343 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488248_T2961s_1.ptu
344 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48895_T1136s_1.ptu
345 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48826_T303s_1.ptu
346 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488191_T2293s_1.ptu
347 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488137_T1634s_1.ptu
348 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488133_T1585s_1.ptu
349 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488216_T2577s_1.ptu
350 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488107_T1274s_1.ptu
351 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488130_T1557s_1.ptu
352 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488120_T1436s_1.ptu
353 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48868_T806s_1.ptu
354 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488125_T1490s_1.ptu
355 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48832_T374s_1.ptu
356 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488119_T1424s_1.ptu
357 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48877_T914s_1.ptu
358 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488188_T2257s_1.ptu
359 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48836_T423s_1.ptu
360 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488176_T2112s_1.ptu
361 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488112_T1334s_1.ptu
362 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48815_T170s_1.ptu
363 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48838_T448s_1.ptu
364 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488178_T2136s_1.ptu
365 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488198_T2363s_1.ptu
366 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48855_T650s_1.ptu
367 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48899_T1178s_1.ptu
368 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488161_T1931s_1.ptu
369 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488246_T2957s_1.ptu
370 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488231_T2757s_1.ptu
371 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488122_T1460s_1.ptu
372 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488157_T1873s_1.ptu
373 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488194_T2329s_1.ptu
374 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48846_T542s_1.ptu
375 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488144_T1718s_1.ptu
376 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488111_T1322s_1.ptu
377 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488143_T1705s_1.ptu
378 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48874_T882s_1.ptu
379 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488124_T1478s_1.ptu
380 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488106_T1268s_1.ptu
381 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48892_T1094s_1.ptu
382 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488115_T1376s_1.ptu
383 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48864_T761s_1.ptu
384 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488239_T2853s_1.ptu
385 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48893_T1106s_1.ptu
386 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488154_T1837s_1.ptu
387 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488105_T1256s_1.ptu
388 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48829_T338s_1.ptu
389 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48871_T845s_1.ptu
390 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48898_T1172s_1.ptu
391 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48880_T954s_1.ptu
392 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48811_T122s_1.ptu
393 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488141_T1682s_1.ptu
394 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488237_T2829s_1.ptu
395 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48810_T110s_1.ptu
396 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48898_T1166s_1.ptu
397 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48812_T135s_1.ptu
398 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48887_T1034s_1.ptu
399 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF488202_T2410s_1.ptu
400 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48828_T326s_1.ptu
401 of 424: /beegfs/ye53nis/data/Pablo_structured_experiment/all_clean_ptu/20 nM AF48886_T1027s_1.ptu
Different binning was chosen for correlation. Loading dataset 1 with bin=100000.0. This can take a while...
Processing correlation of unprocessed dataset 1
Processing correlation with correction by prediction of dataset 1
Loading dataset 2 from path /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu with bin=1e6. This can take a while...
1 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48869_T825s_1.ptu
2 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488196_T2359s_1.ptu
3 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488146_T1755s_1.ptu
4 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488227_T2733s_1.ptu
5 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48823_T287s_1.ptu
6 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48829_T342s_1.ptu
7 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488194_T2334s_1.ptu
8 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48851_T607s_1.ptu
9 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488210_T2528s_1.ptu
10 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488234_T2818s_1.ptu
11 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48882_T982s_1.ptu
12 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488251_T3022s_1.ptu
13 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48848_T610s_1.ptu
14 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488160_T1924s_1.ptu
15 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48846_T584s_1.ptu
16 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488268_T3227s_1.ptu
17 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48823_T268s_1.ptu
18 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48832_T378s_1.ptu
19 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488241_T2902s_1.ptu
20 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488198_T2383s_1.ptu
21 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48869_T823s_1.ptu
22 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48895_T1139s_1.ptu
23 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48878_T934s_1.ptu
24 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488100_T1197s_1.ptu
25 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488155_T1864s_1.ptu
26 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488132_T1585s_1.ptu
27 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48866_T787s_1.ptu
28 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48877_T922s_1.ptu
29 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48861_T728s_1.ptu
30 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488129_T1548s_1.ptu
31 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48825_T292s_1.ptu
32 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488205_T2468s_1.ptu
33 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4889_T106s_1.ptu
34 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48840_T474s_1.ptu
35 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48843_T546s_1.ptu
36 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48887_T1042s_1.ptu
37 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488116_T1391s_1.ptu
38 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4884_T39s_1.ptu
39 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488110_T1318s_1.ptu
40 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488218_T2625s_1.ptu
41 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48870_T834s_1.ptu
42 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488270_T3251s_1.ptu
43 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488177_T2129s_1.ptu
44 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488136_T1633s_1.ptu
45 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48844_T521s_1.ptu
46 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48829_T341s_1.ptu
47 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4887_T75s_1.ptu
48 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48842_T497s_1.ptu
49 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488291_T3505s_1.ptu
50 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488117_T1403s_1.ptu
51 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48814_T159s_1.ptu
52 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488215_T2589s_1.ptu
53 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488128_T1536s_1.ptu
54 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488135_T1621s_1.ptu
55 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48870_T837s_1.ptu
56 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488239_T2878s_1.ptu
57 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48827_T317s_1.ptu
58 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488231_T2781s_1.ptu
59 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488294_T3541s_1.ptu
60 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488247_T2974s_1.ptu
61 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488226_T2721s_1.ptu
62 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488182_T2189s_1.ptu
63 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488248_T2986s_1.ptu
64 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488216_T2601s_1.ptu
65 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48824_T282s_1.ptu
66 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488152_T1827s_1.ptu
67 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48868_T811s_1.ptu
68 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488112_T1342s_1.ptu
69 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48811_T123s_1.ptu
70 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488185_T2225s_1.ptu
71 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48828_T330s_1.ptu
72 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488232_T2793s_1.ptu
73 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488293_T3529s_1.ptu
74 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48825_T294s_1.ptu
75 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4886_T66s_1.ptu
76 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48822_T273s_1.ptu
77 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48889_T1067s_1.ptu
78 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48816_T183s_1.ptu
79 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48879_T946s_1.ptu
80 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48876_T910s_1.ptu
81 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48840_T507s_1.ptu
82 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48849_T583s_1.ptu
83 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48850_T595s_1.ptu
84 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48822_T257s_1.ptu
85 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488184_T2213s_1.ptu
86 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48813_T148s_1.ptu
87 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488121_T1452s_1.ptu
88 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488254_T3059s_1.ptu
89 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48814_T171s_1.ptu
90 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48849_T581s_1.ptu
91 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488122_T1464s_1.ptu
92 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488153_T1839s_1.ptu
93 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48851_T605s_1.ptu
94 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48819_T222s_1.ptu
95 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488104_T1246s_1.ptu
96 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488179_T2153s_1.ptu
97 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488130_T1560s_1.ptu
98 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48874_T885s_1.ptu
99 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488115_T1379s_1.ptu
100 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488282_T3397s_1.ptu
101 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488259_T3119s_1.ptu
102 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48858_T690s_1.ptu
103 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48821_T244s_1.ptu
104 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48825_T313s_1.ptu
105 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48861_T726s_1.ptu
106 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48817_T196s_1.ptu
107 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488265_T3191s_1.ptu
108 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48834_T400s_1.ptu
109 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48854_T643s_1.ptu
110 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48873_T873s_1.ptu
111 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48857_T678s_1.ptu
112 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488208_T2504s_1.ptu
113 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4884_T41s_1.ptu
114 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48810_T120s_1.ptu
115 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48873_T870s_1.ptu
116 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48872_T859s_1.ptu
117 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488180_T2165s_1.ptu
118 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48828_T328s_1.ptu
119 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488224_T2697s_1.ptu
120 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488167_T2009s_1.ptu
121 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488237_T2854s_1.ptu
122 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488258_T3107s_1.ptu
123 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488222_T2673s_1.ptu
124 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48845_T534s_1.ptu
125 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488280_T3373s_1.ptu
126 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48848_T569s_1.ptu
127 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48812_T145s_1.ptu
128 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48844_T522s_1.ptu
129 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48828_T351s_1.ptu
130 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488173_T2081s_1.ptu
131 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48893_T1115s_1.ptu
132 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48865_T774s_1.ptu
133 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48862_T740s_1.ptu
134 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488187_T2250s_1.ptu
135 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48812_T136s_1.ptu
136 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48853_T631s_1.ptu
137 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48843_T509s_1.ptu
138 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48880_T955s_1.ptu
139 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48845_T535s_1.ptu
140 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488133_T1597s_1.ptu
141 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48836_T426s_1.ptu
142 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488138_T1658s_1.ptu
143 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488235_T2830s_1.ptu
144 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488111_T1330s_1.ptu
145 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488156_T1876s_1.ptu
146 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48855_T655s_1.ptu
147 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4887_T80s_1.ptu
148 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48831_T365s_1.ptu
149 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48854_T642s_1.ptu
150 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48872_T861s_1.ptu
151 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488107_T1282s_1.ptu
152 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48824_T280s_1.ptu
153 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488256_T3083s_1.ptu
154 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488204_T2456s_1.ptu
155 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488197_T2371s_1.ptu
156 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48852_T619s_1.ptu
157 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488176_T2117s_1.ptu
158 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488229_T2757s_1.ptu
159 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48842_T498s_1.ptu
160 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48826_T305s_1.ptu
161 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48864_T762s_1.ptu
162 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488289_T3481s_1.ptu
163 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48849_T624s_1.ptu
164 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488249_T2998s_1.ptu
165 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48816_T184s_1.ptu
166 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48817_T197s_1.ptu
167 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48884_T1006s_1.ptu
168 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488178_T2141s_1.ptu
169 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48855_T654s_1.ptu
170 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48813_T158s_1.ptu
171 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488118_T1415s_1.ptu
172 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488127_T1524s_1.ptu
173 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48836_T425s_1.ptu
174 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48830_T352s_1.ptu
175 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48829_T340s_1.ptu
176 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488261_T3143s_1.ptu
177 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48837_T468s_1.ptu
178 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48830_T354s_1.ptu
179 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48819_T220s_1.ptu
180 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488236_T2842s_1.ptu
181 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488257_T3095s_1.ptu
182 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48819_T219s_1.ptu
183 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488253_T3047s_1.ptu
184 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48841_T485s_1.ptu
185 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48830_T377s_1.ptu
186 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488201_T2420s_1.ptu
187 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48835_T412s_1.ptu
188 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488269_T3239s_1.ptu
189 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48846_T545s_1.ptu
190 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48833_T416s_1.ptu
191 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48839_T461s_1.ptu
192 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48850_T637s_1.ptu
193 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488276_T3324s_1.ptu
194 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488286_T3445s_1.ptu
195 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488181_T2177s_1.ptu
196 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488199_T2395s_1.ptu
197 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488244_T2938s_1.ptu
198 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488101_T1209s_1.ptu
199 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48881_T970s_1.ptu
200 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488126_T1512s_1.ptu
201 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488223_T2685s_1.ptu
202 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48847_T557s_1.ptu
203 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488158_T1900s_1.ptu
204 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488139_T1670s_1.ptu
205 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48824_T281s_1.ptu
206 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48859_T702s_1.ptu
207 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48846_T546s_1.ptu
208 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488267_T3215s_1.ptu
209 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4883_T28s_1.ptu
210 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48827_T318s_1.ptu
211 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48820_T232s_1.ptu
212 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48860_T716s_1.ptu
213 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4882_T15s_1.ptu
214 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48896_T1151s_1.ptu
215 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48830_T353s_1.ptu
216 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488262_T3155s_1.ptu
217 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48899_T1187s_1.ptu
218 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488209_T2516s_1.ptu
219 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488150_T1803s_1.ptu
220 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488147_T1767s_1.ptu
221 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488134_T1609s_1.ptu
222 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48815_T171s_1.ptu
223 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488264_T3179s_1.ptu
224 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488298_T3590s_1.ptu
225 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488193_T2322s_1.ptu
226 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4888_T87s_1.ptu
227 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488106_T1270s_1.ptu
228 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488284_T3421s_1.ptu
229 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488145_T1743s_1.ptu
230 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48828_T329s_1.ptu
231 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488183_T2201s_1.ptu
232 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48856_T668s_1.ptu
233 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4888_T93s_1.ptu
234 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4889_T100s_1.ptu
235 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48818_T222s_1.ptu
236 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48835_T442s_1.ptu
237 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488175_T2105s_1.ptu
238 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48841_T486s_1.ptu
239 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48833_T390s_1.ptu
240 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488245_T2950s_1.ptu
241 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488195_T2347s_1.ptu
242 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48867_T799s_1.ptu
243 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48839_T494s_1.ptu
244 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48817_T195s_1.ptu
245 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488105_T1258s_1.ptu
246 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48897_T1163s_1.ptu
247 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48835_T413s_1.ptu
248 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48820_T248s_1.ptu
249 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48841_T520s_1.ptu
250 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4881_T0s_1.ptu
251 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488165_T1984s_1.ptu
252 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48887_T1040s_1.ptu
253 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48817_T210s_1.ptu
254 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4883_T27s_1.ptu
255 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48891_T1091s_1.ptu
256 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488233_T2806s_1.ptu
257 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48811_T132s_1.ptu
258 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48876_T906s_1.ptu
259 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48839_T462s_1.ptu
260 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488109_T1306s_1.ptu
261 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488162_T1948s_1.ptu
262 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48858_T692s_1.ptu
263 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488151_T1815s_1.ptu
264 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488203_T2444s_1.ptu
265 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48815_T184s_1.ptu
266 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48831_T366s_1.ptu
267 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48883_T991s_1.ptu
268 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48881_T967s_1.ptu
269 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488287_T3457s_1.ptu
270 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48884_T1003s_1.ptu
271 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48886_T1031s_1.ptu
272 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48820_T234s_1.ptu
273 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48845_T533s_1.ptu
274 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488100_T1199s_1.ptu
275 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488214_T2577s_1.ptu
276 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48845_T571s_1.ptu
277 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48856_T666s_1.ptu
278 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488172_T2069s_1.ptu
279 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48837_T438s_1.ptu
280 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48822_T256s_1.ptu
281 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488243_T2926s_1.ptu
282 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488240_T2890s_1.ptu
283 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488206_T2480s_1.ptu
284 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48818_T208s_1.ptu
285 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48866_T788s_1.ptu
286 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48863_T752s_1.ptu
287 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488238_T2866s_1.ptu
288 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488260_T3131s_1.ptu
289 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48818_T210s_1.ptu
290 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48892_T1103s_1.ptu
291 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48813_T147s_1.ptu
292 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48834_T402s_1.ptu
293 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48853_T630s_1.ptu
294 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488114_T1367s_1.ptu
295 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488300_T3614s_1.ptu
296 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488119_T1427s_1.ptu
297 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48868_T813s_1.ptu
298 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488157_T1888s_1.ptu
299 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488188_T2262s_1.ptu
300 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488212_T2552s_1.ptu
301 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48833_T388s_1.ptu
302 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48864_T764s_1.ptu
303 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48823_T269s_1.ptu
304 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488252_T3035s_1.ptu
305 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488169_T2032s_1.ptu
306 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48890_T1079s_1.ptu
307 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48871_T847s_1.ptu
308 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488137_T1646s_1.ptu
309 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488166_T1997s_1.ptu
310 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488283_T3409s_1.ptu
311 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488279_T3360s_1.ptu
312 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488125_T1500s_1.ptu
313 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488200_T2408s_1.ptu
314 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48882_T979s_1.ptu
315 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488281_T3385s_1.ptu
316 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48811_T124s_1.ptu
317 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488102_T1221s_1.ptu
318 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48825_T293s_1.ptu
319 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488186_T2237s_1.ptu
320 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488274_T3299s_1.ptu
321 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48838_T450s_1.ptu
322 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48826_T304s_1.ptu
323 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488163_T1960s_1.ptu
324 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488174_T2093s_1.ptu
325 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488250_T3010s_1.ptu
326 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488255_T3071s_1.ptu
327 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48847_T597s_1.ptu
328 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488288_T3469s_1.ptu
329 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488103_T1234s_1.ptu
330 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488164_T1972s_1.ptu
331 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48885_T1018s_1.ptu
332 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48821_T246s_1.ptu
333 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48812_T135s_1.ptu
334 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488242_T2914s_1.ptu
335 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488142_T1706s_1.ptu
336 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488159_T1912s_1.ptu
337 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4887_T76s_1.ptu
338 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48886_T1028s_1.ptu
339 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488263_T3167s_1.ptu
340 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48824_T299s_1.ptu
341 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488113_T1354s_1.ptu
342 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488108_T1294s_1.ptu
343 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48816_T197s_1.ptu
344 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48863_T750s_1.ptu
345 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488140_T1682s_1.ptu
346 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488230_T2769s_1.ptu
347 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488131_T1573s_1.ptu
348 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488220_T2649s_1.ptu
349 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48838_T449s_1.ptu
350 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488154_T1852s_1.ptu
351 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48838_T481s_1.ptu
352 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488120_T1439s_1.ptu
353 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488207_T2492s_1.ptu
354 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488171_T2056s_1.ptu
355 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488190_T2286s_1.ptu
356 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48847_T558s_1.ptu
357 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48898_T1175s_1.ptu
358 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48862_T738s_1.ptu
359 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48816_T186s_1.ptu
360 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488299_T3602s_1.ptu
361 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488219_T2637s_1.ptu
362 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48810_T111s_1.ptu
363 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488278_T3348s_1.ptu
364 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488189_T2274s_1.ptu
365 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48859_T704s_1.ptu
366 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48832_T376s_1.ptu
367 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48865_T776s_1.ptu
368 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488225_T2709s_1.ptu
369 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48871_T849s_1.ptu
370 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48883_T994s_1.ptu
371 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48831_T390s_1.ptu
372 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488272_T3276s_1.ptu
373 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488148_T1779s_1.ptu
374 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48875_T898s_1.ptu
375 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48842_T532s_1.ptu
376 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488290_T3493s_1.ptu
377 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48877_T918s_1.ptu
378 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488297_T3578s_1.ptu
379 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488191_T2298s_1.ptu
380 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48814_T160s_1.ptu
381 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48844_T558s_1.ptu
382 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488221_T2661s_1.ptu
383 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488211_T2540s_1.ptu
384 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48821_T260s_1.ptu
385 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488123_T1475s_1.ptu
386 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48852_T617s_1.ptu
387 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488149_T1791s_1.ptu
388 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488141_T1694s_1.ptu
389 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48827_T338s_1.ptu
390 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488217_T2613s_1.ptu
391 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48848_T571s_1.ptu
392 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4886_T63s_1.ptu
393 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48843_T510s_1.ptu
394 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48810_T112s_1.ptu
395 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488124_T1488s_1.ptu
396 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF488271_T3263s_1.ptu
397 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48840_T473s_1.ptu
398 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48826_T306s_1.ptu
399 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF4885_T54s_1.ptu
400 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48827_T316s_1.ptu
401 of 440: /beegfs/ye53nis/data/Pablo_structured_experiment/all_dirty_ptu/DiO LUV 10uM in 20 nM AF48850_T593s_1.ptu
Different binning was chosen for correlation. Loading dataset 2 with bin=100000.0. This can take a while...
Processing correlation of unprocessed dataset 2
Processing correlation with correction by prediction of dataset 2
/beegfs/ye53nis/drmed-git/src/fluotracify/applications/correction.py:508: UserWarning: Metadata is not saved with data. Reason: the correlation algorithm failed for one or more traces which were shorter than 32 time steps after correction.Since metadata is loaded in the beginning, it is not sure, which correlation is missing to ensure proper joining of data and metadata.
  warnings.warn('Metadata is not saved with data. Reason: the '

corr_out = pd.read_csv(filepath_or_buffer='data/exp-210807-hparams/2021-08-26_correlations.csv')
corr_out
  \(D\) in \(\frac{{\mu m^2}}{{s}}\) \(\tau_{{D}}\) in \(ms\) Trace lengths folderid-tracesused Photon count bin for correlation in \(ns\)
0 19.078159 0.590783 8192.0 0-orig 100000.0
1 20.314897 0.554817 8192.0 0-orig 100000.0
2 21.007772 0.536518 8192.0 0-orig 100000.0
3 22.716317 0.496166 8192.0 0-orig 100000.0
4 24.089236 0.467888 8192.0 0-orig 100000.0
… … … … … …
4795 0.577566 19.514749 7240.0 1-pred-0.9 100000.0
4796 0.236147 47.728941 7078.0 1-pred-0.9 100000.0
4797 1.860295 6.058745 7819.0 1-pred-0.9 100000.0
4798 1.946114 5.791570 7520.0 1-pred-0.9 100000.0
4799 0.859799 13.108948 7841.0 1-pred-0.9 100000.0

4800 rows × 5 columns

  1. Check out the nan values 
    corr_out[corr_out['$D$ in $\\frac{{\mu m^2}}{{s}}$'].isna()]

    \(D\) in \(\frac{{\mu m^2}}{{s}}\) \(\tau_{{D}}\) in \(ms\) Trace lengths folderid-tracesused Photon count bin for correlation in \(ns\)

    Good news! This time there are no nan values. In exp-210204-unet, we had 31 of them in the clean dataset with pred-0.1, which suggested a lot of false positives in the clean data, which led to traces shorter than 32 time steps, where the multipletau algorithm fails. Still, I am not sure why the metadata was not saved out in the correlation program. I will investigate another time…

  2. Refactor folder-id_traces-used into 2 columns 
    corr_out['folder_id-traces_used']

    0           0-orig
1           0-orig
2           0-orig
3           0-orig
4           0-orig
           ...
4795    1-pred-0.9
4796    1-pred-0.9
4797    1-pred-0.9
4798    1-pred-0.9
4799    1-pred-0.9
Name: folder_id-traces_used, Length: 4800, dtype: object

    corr_out[['Folder ID', 'Traces used']] = corr_out['folder_id-traces_used'].str.split(pat='-', n=1, expand=True)
  3. Some basic statistical descriptions 
    corr_stats = pd.DataFrame()
for fid in sorted(set(corr_out['Folder ID'])):
    for tu in sorted(set(corr_out['Traces used'])):
        corr_new = corr_out[(corr_out['Folder ID'] == fid) & (corr_out['Traces used'] == tu)].describe()
        new_index_tuple = zip(((fid),)*8, ((tu),)*8, ('count', 'mean', 'std', 'min', '25%', '50%', '75%', 'max'))
        corr_new.index = pd.MultiIndex.from_tuples(new_index_tuple, names=['Folder ID', 'Traces used', 'Stats'])
        corr_stats = pd.concat([corr_stats, corr_new], axis=0)
with pd.option_context('display.max_rows', None, 'display.max_columns', None):
    display(corr_stats)
    $D$ in $\frac{{\mu m^2}}{{s}}$ $\tau_{{D}}$ in $ms$ Trace lengths Photon count bin for correlation in $ns$
    Folder ID Traces used Stats
    0 orig count 400.000000 400.000000 400.000000 400.0
    mean 22.397754 0.507013 8192.000000 100000.0
    std 1.994998 0.042972 0.000000 0.0
    min 18.207878 0.379407 8192.000000 100000.0
    25% 20.921029 0.481523 8192.000000 100000.0
    50% 22.149393 0.508866 8192.000000 100000.0
    75% 23.407135 0.538743 8192.000000 100000.0
    max 29.707022 0.619021 8192.000000 100000.0
    pred-0.1 count 400.000000 400.000000 400.000000 400.0
    mean 22.444596 0.502583 8182.360000 100000.0
    std 0.645526 0.014330 8.417990 0.0
    min 20.681787 0.460807 8124.000000 100000.0
    25% 21.961521 0.492386 8179.000000 100000.0
    50% 22.368956 0.503870 8184.000000 100000.0
    75% 22.890670 0.513218 8188.000000 100000.0
    max 24.459398 0.544975 8192.000000 100000.0
    pred-0.3 count 400.000000 400.000000 400.000000 400.0
    mean 22.420126 0.503131 8190.077500 100000.0
    std 0.644112 0.014326 2.892648 0.0
    min 20.668056 0.461330 8174.000000 100000.0
    25% 21.946382 0.493151 8189.000000 100000.0
    50% 22.342680 0.504463 8191.000000 100000.0
    75% 22.855193 0.513572 8192.000000 100000.0
    max 24.431668 0.545337 8192.000000 100000.0
    pred-0.5 count 400.000000 400.000000 400.000000 400.0
    mean 22.411363 0.503324 8191.300000 100000.0
    std 0.641300 0.014276 1.488469 0.0
    min 20.666124 0.462115 8183.000000 100000.0
    25% 21.944307 0.493346 8191.000000 100000.0
    50% 22.325232 0.504857 8192.000000 100000.0
    75% 22.846137 0.513621 8192.000000 100000.0
    max 24.390149 0.545388 8192.000000 100000.0
    pred-0.7 count 400.000000 400.000000 400.000000 400.0
    mean 22.403669 0.503500 8191.810000 100000.0
    std 0.643185 0.014327 0.797175 0.0
    min 20.615435 0.462115 8184.000000 100000.0
    25% 21.935997 0.493728 8192.000000 100000.0
    50% 22.323219 0.504903 8192.000000 100000.0
    75% 22.828459 0.513816 8192.000000 100000.0
    max 24.390149 0.546729 8192.000000 100000.0
    pred-0.9 count 400.000000 400.000000 400.000000 400.0
    mean 22.401607 0.503547 8191.937500 100000.0
    std 0.643493 0.014338 0.509158 0.0
    min 20.615435 0.462115 8185.000000 100000.0
    25% 21.917571 0.493728 8192.000000 100000.0
    50% 22.323219 0.504903 8192.000000 100000.0
    75% 22.828459 0.514247 8192.000000 100000.0
    max 24.390149 0.546729 8192.000000 100000.0
    1 orig count 400.000000 400.000000 400.000000 400.0
    mean 2.089838 106.604322 8192.000000 100000.0
    std 4.867448 244.689593 0.000000 0.0
    min 0.005636 0.370469 8192.000000 100000.0
    25% 0.110612 9.232342 8192.000000 100000.0
    50% 0.277676 40.594163 8192.000000 100000.0
    75% 1.220995 101.904123 8192.000000 100000.0
    max 30.423708 2000.000000 8192.000000 100000.0
    pred-0.1 count 400.000000 400.000000 400.000000 400.0
    mean 23.487424 0.729772 6014.930000 100000.0
    std 11.632062 0.978357 655.329984 0.0
    min 0.704858 0.213925 4055.000000 100000.0
    25% 14.296784 0.349318 5582.750000 100000.0
    50% 22.055818 0.511024 6069.500000 100000.0
    75% 32.266393 0.788363 6477.250000 100000.0
    max 52.686940 15.990532 7586.000000 100000.0
    pred-0.3 count 400.000000 400.000000 400.000000 400.0
    mean 16.200632 1.809016 6649.735000 100000.0
    std 10.841992 4.819835 535.204704 0.0
    min 0.204291 0.221455 4702.000000 100000.0
    25% 7.332725 0.495671 6317.250000 100000.0
    50% 14.302030 0.788074 6699.500000 100000.0
    75% 22.739144 1.537091 7043.250000 100000.0
    max 50.895517 55.171591 7781.000000 100000.0
    pred-0.5 count 400.000000 400.000000 400.000000 400.0
    mean 10.787920 4.217836 7002.527500 100000.0
    std 8.761952 11.682295 448.347782 0.0
    min 0.080967 0.273684 5168.000000 100000.0
    25% 3.713476 0.730534 6733.500000 100000.0
    50% 8.729210 1.291190 7049.000000 100000.0
    75% 15.428534 3.035206 7327.750000 100000.0
    max 41.182749 139.205234 7887.000000 100000.0
    pred-0.7 count 400.000000 400.000000 400.000000 400.0
    mean 6.669685 9.416059 7267.402500 100000.0
    std 6.356652 21.185204 368.356589 0.0
    min 0.065038 0.375238 5715.000000 100000.0
    25% 1.657642 1.154375 7053.500000 100000.0
    50% 4.786510 2.355174 7323.000000 100000.0
    75% 9.763811 6.803933 7543.250000 100000.0
    max 30.037104 173.299931 7969.000000 100000.0
    pred-0.9 count 400.000000 400.000000 400.000000 400.0
    mean 3.074691 21.394545 7556.762500 100000.0
    std 3.639749 36.676351 269.188535 0.0
    min 0.038601 0.593323 6349.000000 100000.0
    25% 0.459359 2.485594 7406.750000 100000.0
    50% 1.544961 7.295842 7603.500000 100000.0
    75% 4.534553 24.536530 7750.000000 100000.0
    max 18.996487 291.988834 8072.000000 100000.0
  4. Have a look at the trace lengths 
    x = 'Trace lengths'

g = sns.FacetGrid(data=corr_out,
                  col='Folder ID',
                  col_wrap=1,
                  sharex=True,
                  aspect=1.5,
                  height=5,
                  legend_out=True)
g.map_dataframe(sns.boxplot,
      x=x,
      y='Traces used',
      palette='colorblind',
      showfliers=False)
g.map_dataframe(sns.stripplot,
      x=x,
      y='Traces used',
      dodge=True,
      palette=sns.color_palette(['0.3']),
      size=4,
      jitter=0.2)
g.set_xlabels(x)
g.fig.patch.set_facecolor('white')
g.tight_layout()
plt.show()

    Trace lengths: 2021-08-26_correlations_tracelength.png

    First of all: the extension of simulated training data clearly helped to avoid the enormous amount of false positives in the run exp-210204-unet in folder 0. Here, only few traces are shortened, which is to be expected, since it represents a negative control. Looking at folder 1, the reduction of trace length increased with all prediction thresholds. I later plot trace length vs tau to see if this reduction in trace length introduces artifacts or if it is mainly a desired outcome of more found artifacts.

  5. Have a look at diffusion coefficient and transit times 
    # '$D$ in $\\frac{{\mu m^2}}{{s}}$'
# '$\\tau_{{D}}$ in $ms$'
x = '$D$ in $\\frac{{\mu m^2}}{{s}}$'

g = sns.FacetGrid(data=corr_out,
                  col='Folder ID',
                  col_wrap=1,
                  sharex=True,
                  aspect=1.5,
                  height=5,
                  legend_out=True)
g.map_dataframe(sns.boxplot,
      x=x,
      y='Traces used',
      palette='colorblind',
      showfliers=False).set(xscale = 'log')
g.map_dataframe(sns.stripplot,
      x=x,
      y='Traces used',
      dodge=True,
      palette=sns.color_palette(['0.3']),
      size=4,
      jitter=0.2).set(xscale = 'log')
g.set_xlabels(x)
g.tight_layout()
g.fig.patch.set_facecolor('white')
plt.show()

    tau: 2021-08-26_correlations_tau.png D: 2021-08-26_correlations_diffrate.png

  6. This scatterplot shows Diffusion rates / transit times against trace lengths. I used a subsample to avoid overplotting. 
    # '$D$ in $\\frac{{\mu m^2}}{{s}}$'
# '$\\tau_{{D}}$ in $ms$'
x = '$\\tau_{{D}}$ in $ms$'

g = sns.FacetGrid(data=corr_out,
                  row='Traces used',
                  col='Folder ID',
                  hue='Traces used',
                  sharex=True,
                  sharey=True,
                  aspect=1.5,
                  height=3.5,
                  margin_titles=True,
                  legend_out=True)
g.map_dataframe(sns.scatterplot,
      x=x,
      y='Trace lengths',
      palette='colorblind').set(xscale = 'log')
g.add_legend(title='Traces used')
g.set_xlabels(x)
g.tight_layout()
g.fig.patch.set_facecolor('white')
plt.show()

    2021-08-26_correlations_tlvstau.png 

    # '$D$ in $\\frac{{\mu m^2}}{{s}}$'
# '$\\tau_{{D}}$ in $ms$'
x = '$\\tau_{{D}}$ in $ms$'

g = sns.FacetGrid(data=corr_out.sample(1000),
                  row='Folder ID',
                  hue='Traces used',
                  hue_order=['orig', 'pred-0.1', 'pred-0.3', 'pred-0.5', 'pred-0.7', 'pred-0.9'],
                  sharex=True,
                  sharey=True,
                  aspect=1.5,
                  height=4,
                  margin_titles=True,
                  legend_out=True)
g.map_dataframe(sns.scatterplot,
      x=x,
      y='Trace lengths',
      palette='colorblind').set(xscale = 'log')
g.add_legend(title='Traces used')
g.set_xlabels(x)
g.tight_layout()
g.fig.patch.set_facecolor('white')
plt.show()

    2021-08-26_correlations_tlvstau2.png

    For me, this plot shows that the reduction of trace length is not too severe even with low prediction thresholds. Also, the reduction in trace length mainly seems to be due to better recognition of artifacts.

2.1.7 Apply model from run 2 to simulated data

%cd /beegfs/ye53nis/drmed-git

/beegfs/ye53nis/drmed-git

from pathlib import Path

import sys
import mlflow
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
import tensorflow as tf
print("tf version: ", tf.version.VERSION)
print("tf.keras version: ", tf.keras.__version__)
print("mlflow version: ", mlflow.version.VERSION)

tf version:  2.5.0
tf.keras version:  2.5.0
mlflow version:  1.19.0

sys.path.append('src/')
from fluotracify.simulations import (
   import_simulation_from_csv as isfc,
   analyze_simulations as ans,
)
from fluotracify.training import build_model as bm, preprocess_data as ppd
from fluotracify.applications import correlate, plots, correction
from fluotracify.imports import ptu_utils as ptu

folder = '/beegfs/ye53nis/saves/firstartifact_Nov2020_test'
col_per_example = 3
lab_thresh = 0.04
pred_thresh = 0.5
artifact = 0
model_type = 1
fwhm = 250
logged_model = 'file:///beegfs/ye53nis/drmed-git/data/mlruns/8/00f2635d9fa2463c9a066722163405be/artifacts/model'
metrics_thresholds = [0.1, 0.3, 0.5, 0.7, 0.9]

loaded_model = mlflow.keras.load_model(logged_model, compile=False)
loaded_model.compile(loss=bm.binary_ce_dice_loss(),
                     optimizer=tf.keras.optimizers.Adam(),
                     metrics = bm.unet_metrics(metrics_thresholds))
loaded_model

WARNING:tensorflow:Unable to restore custom metric. Please ensure that the layer implements `get_config` and `from_config` when saving. In addition, please use the `custom_objects` arg when calling `load_model()`.
<tensorflow.python.keras.engine.functional.Functional at 0x2b01891b62e0>

# The following block has to be executed multiple times with different array lengths
# e.g. 2**14, 2**13, 2**12
# note that: after executing 2**14, then 2**13, there will be an error
# if you then execute 2**12 and then 2**13 again, the error will be gone
test_features = np.zeros((2**13)).reshape(1, -1, 1)
print(test_features.shape)
predictions = loaded_model.predict(test_features, verbose=0).flatten()
predictions

(1, 8192, 1)
array([1.4064237e-01, 1.3650321e-08, 7.4799689e-10, ..., 3.1755112e-10,
       1.8236226e-10, 6.6588524e-10], dtype=float32)

The simulated data was separated in train, val, and test beforehand. Now, I only load test data. 

dataset, _, nsamples, experiment_params = isfc.import_from_csv(
    folder=folder,
    header=12,
    frac_train=1,
    col_per_example=col_per_example,
    dropindex=None,
    dropcolumns=None)

0 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.069/1.0/traces_brightclust_Nov2020_D0.069_set010.csv
1 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/50/0.1/traces_brightclust_Nov2020_D50_set003.csv
2 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.4/0.1/traces_brightclust_Nov2020_D0.4_set001.csv
3 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.2/0.1/traces_brightclust_Nov2020_D0.2_set007.csv
4 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/3.0/1.0/traces_brightclust_Nov2020_D3.0_set002.csv
5 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/3.0/0.01/traces_brightclust_Nov2020_D3.0_set004.csv
6 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/50/0.01/traces_brightclust_Nov2020_D50_set002.csv
7 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.2/1.0/traces_brightclust_Nov2020_D0.2_set005.csv
8 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.6/1.0/traces_brightclust_Nov2020_D0.6_set009.csv
9 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/10/0.1/traces_brightclust_Nov2020_D10_set001.csv10 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.08/1.0/traces_brightclust_Nov2020_D0.08_set001.csv
11 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.6/0.1/traces_brightclust_Nov2020_D0.6_set003.csv
12 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.1/1.0/traces_brightclust_Nov2020_D0.1_set001.csv
13 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.4/1.0/traces_brightclust_Nov2020_D0.4_set005.csv
14 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/10/1.0/traces_brightclust_Nov2020_D10_set005.csv
15 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/1.0/1.0/traces_brightclust_Nov2020_D1.0_set005.csv
16 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.069/0.1/traces_brightclust_Nov2020_D0.069_set001.csv
17 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/50/1.0/traces_brightclust_Nov2020_D50_set001.csv
18 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.1/0.01/traces_brightclust_Nov2020_D0.1_set002.csv
19 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.08/0.1/traces_brightclust_Nov2020_D0.08_set003.csv
20 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/1.0/0.01/traces_brightclust_Nov2020_D1.0_set006.csv
21 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/1.0/0.1/traces_brightclust_Nov2020_D1.0_set003.csv
22 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.2/0.01/traces_brightclust_Nov2020_D0.2_set002.csv
23 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.1/0.1/traces_brightclust_Nov2020_D0.1_set005.csv
24 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/3.0/0.1/traces_brightclust_Nov2020_D3.0_set007.csv
25 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.08/0.01/traces_brightclust_Nov2020_D0.08_set005.csv
26 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.069/0.01/traces_brightclust_Nov2020_D0.069_set005.csv
27 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/10/0.01/traces_brightclust_Nov2020_D10_set002.csv
28 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.6/0.01/traces_brightclust_Nov2020_D0.6_set008.csv
29 /beegfs/ye53nis/saves/firstartifact_Nov2020_test/0.4/0.01/traces_brightclust_Nov2020_D0.4_set008.csv

experiment_params.T.sort_values(by='diffusion rate of molecules in micrometer^2 / s', ignore_index=True)
  unique identifier path and file name FWHMs of excitation PSFs used in nm Extent of simulated PSF (distance to center of Gaussian) in nm total simulation time in ms time step in ms number of fast molecules diffusion rate of molecules in micrometer2 / s width of the simulation in nm height of the simulation in nm number of slow clusters diffusion rate of clusters in micrometer2 / s trace001
0 bebf4d6b-0ffe-4dcf-be78-93bfa70dd04c /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 679 0.069 3000.0 3000.0 3 1.0 label0011
1 5abeceef-c30b-46d8-96a6-dfd623820542 /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 1930 0.069 3000.0 3000.0 10 0.01 label0011
2 daf0898c-38d3-48e5-8d97-41490fb30cd7 /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 897 0.069 3000.0 3000.0 7 0.1 label0011
3 cae1a0a7-6d83-49fe-b0b7-fb1f7fa3d541 /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 1673 0.08 3000.0 3000.0 10 0.01 label0011
4 b85aeec3-5e79-444f-b0c5-5ba7057fad8c /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 856 0.08 3000.0 3000.0 7 0.1 label0011
5 9f7d64ef-5e20-4a2a-bf66-adefc770d2d3 /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 2569 0.08 3000.0 3000.0 3 1.0 label0011
6 0d5dcb67-7e31-43dd-8036-11b7243daf14 /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 1156 0.1 3000.0 3000.0 7 0.1 label0011
7 9a2f89bd-c64e-4324-910c-46a5b8a657ba /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 2975 0.1 3000.0 3000.0 10 0.01 label0011
8 db48ccc0-e770-4570-8cf0-d0ea83bfc588 /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 1456 0.1 3000.0 3000.0 3 1.0 label0011
9 33ee5002-7064-4882-88de-79c773d741cd /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 895 0.2 3000.0 3000.0 7 0.1 label0011
10 e3fd87db-7a9e-4f0a-a7f3-30acd95752eb /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 1844 0.2 3000.0 3000.0 10 0.01 label0011
11 511e911a-aaa3-4d9d-a14e-d172bd4d8dbb /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 2735 0.2 3000.0 3000.0 3 1.0 label0011
12 eb3f3035-955f-4e86-bfbd-16421eede63f /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 560 0.4 3000.0 3000.0 3 1.0 label0011
13 20f0a9df-2c4d-409f-86de-ceda40d1aafe /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 1724 0.4 3000.0 3000.0 10 0.01 label0011
14 2c825894-5e96-4acb-9d4d-c6d0c21e71bf /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 1939 0.4 3000.0 3000.0 7 0.1 label0011
15 dd3aef12-414f-4cb7-9330-8db192d67143 /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 2545 0.6 3000.0 3000.0 10 0.01 label0011
16 87d9374a-3ae4-4dfb-8dc3-ec401df4961d /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 1296 0.6 3000.0 3000.0 3 1.0 label0011
17 f43754f2-e756-405f-89a4-9eae43dbf996 /beegfs/ye53nis/saves/firstartifactNov2020/0…. [250] 4000 16384 1.0 2535 0.6 3000.0 3000.0 7 0.1 label0011
18 84116b56-7264-4f63-833a-2fa1af968288 /beegfs/ye53nis/saves/firstartifactNov2020/1…. [250] 4000 16384 1.0 2059 1.0 3000.0 3000.0 3 1.0 label0011
19 c1a85eb4-f202-40f7-8bce-cd3fd3a843dc /beegfs/ye53nis/saves/firstartifactNov2020/1…. [250] 4000 16384 1.0 1357 1.0 3000.0 3000.0 10 0.01 label0011
20 e0118d21-73cf-4c2d-bc3f-f48a44e1b5b8 /beegfs/ye53nis/saves/firstartifactNov2020/1…. [250] 4000 16384 1.0 627 1.0 3000.0 3000.0 7 0.1 label0011
21 6a3c00e3-24e6-4758-9c6d-f2a9dc23f7d9 /beegfs/ye53nis/saves/firstartifactNov2020/10… [250] 4000 16384 1.0 1376 10 3000.0 3000.0 7 0.1 label0011
22 9246b720-b8d4-4390-82b6-71404ced9bea /beegfs/ye53nis/saves/firstartifactNov2020/10… [250] 4000 16384 1.0 1396 10 3000.0 3000.0 10 0.01 label0011
23 e26d94e4-c2d0-46da-b571-91b8fba15f0a /beegfs/ye53nis/saves/firstartifactNov2020/10… [250] 4000 16384 1.0 2618 10 3000.0 3000.0 3 1.0 label0011
24 cf57c545-933f-41b8-bcfe-19723d79b7bf /beegfs/ye53nis/saves/firstartifactNov2020/3…. [250] 4000 16384 1.0 1785 3.0 3000.0 3000.0 10 0.01 label0011
25 f2dcf7ee-f180-4b26-9706-526ceb6b9fe6 /beegfs/ye53nis/saves/firstartifactNov2020/3…. [250] 4000 16384 1.0 2900 3.0 3000.0 3000.0 3 1.0 label0011
26 4f56e86f-7912-4308-b273-1d2cbe919555 /beegfs/ye53nis/saves/firstartifactNov2020/3…. [250] 4000 16384 1.0 3042 3.0 3000.0 3000.0 7 0.1 label0011
27 4a6c04ea-6d95-4c9a-8985-4e2845cd4372 /beegfs/ye53nis/saves/firstartifactNov2020/50… [250] 4000 16384 1.0 3083 50 3000.0 3000.0 3 1.0 label0011
28 0953132e-d7d6-44bb-b1ee-df54b288beb0 /beegfs/ye53nis/saves/firstartifactNov2020/50… [250] 4000 16384 1.0 2323 50 3000.0 3000.0 10 0.01 label0011
29 b2b78a0f-278b-4b7d-82a3-5e625fbc91de /beegfs/ye53nis/saves/firstartifactNov2020/50… [250] 4000 16384 1.0 2561 50 3000.0 3000.0 7 0.1 label0011 
diffrates = experiment_params.loc['diffusion rate of molecules in micrometer^2 / s'].astype(np.float32)
nmols = experiment_params.loc['number of fast molecules'].astype(np.float32)
clusters = experiment_params.loc['diffusion rate of clusters in micrometer^2 / s'].astype(np.float32)

dataset_sep = isfc.separate_data_and_labels(array=dataset,
                                            nsamples=nsamples,
                                            col_per_example=col_per_example)

features = dataset_sep['0']
labels_artifact = dataset_sep['1']
labels_artifact_bool = labels_artifact > lab_thresh
labels_puretrace = dataset_sep['2']

The given DataFrame was split into 3 parts with shapes: [(16384, 3000), (16384, 3000), (16384, 3000)]

Let's correct the traces with the new model and correlate them! 

  corr_out = ans.correlate_simulations_corrected_by_prediction(
      model=loaded_model,
      lab_thresh=lab_thresh,
      pred_thresh=pred_thresh,
      artifact=artifact,
      model_type=model_type,
      experiment_params=experiment_params,
      nsamples=nsamples,
      features=features,
      labels_artifact=labels_artifact,
      labels_puretrace=labels_puretrace,
      save_as_csv=True)
corr_out

processed correlation of 3000 traces with correction by label
processed correlation of 3000 traces with correction by prediction
processed correlation of 3000 traces without correction
processed correlation of pure 3000 traces

corr_out = pd.read_csv(filepath_or_buffer='data/exp-210807-hparams/2021-12-15_correlations.csv')
corr_out
  Simulated \(D\) Simulated \(D_{{clust}}\) nmol \(D\) in \(\frac{{\mu m^2}}{{s}}\) \(\tau_{{D}}\) in \(ms\) Trace lengths Traces used
0 0.069 1.00 679.0 1.037836 10.860146 16384 corrupted without correction
1 0.069 1.00 679.0 0.814030 13.846002 16384 corrupted without correction
2 0.069 1.00 679.0 1.053716 10.696482 16384 corrupted without correction
3 0.069 1.00 679.0 0.897022 12.564974 16384 corrupted without correction
4 0.069 1.00 679.0 1.083577 10.401707 16384 corrupted without correction
… … … … … … … …
11995 0.400 0.01 1724.0 0.337877 33.358455 16267 corrected by prediction
11996 0.400 0.01 1724.0 0.317319 35.519585 7942 corrected by prediction
11997 0.400 0.01 1724.0 0.380969 29.585216 5315 corrected by prediction
11998 0.400 0.01 1724.0 0.376623 29.926600 12591 corrected by prediction
11999 0.400 0.01 1724.0 0.167653 67.228585 13305 corrected by prediction

12000 rows × 7 columns

  1. Check out the NaN values. 
    print('All NaN values: {}'.format(corr_out[corr_out['$D$ in $\\frac{{\mu m^2}}{{s}}$'].isna()]))

    All NaN values: Empty DataFrame
Columns: [Simulated $D$, Simulated $D_{{clust}}$, nmol, $D$ in $\frac{{\mu m^2}}{{s}}$, $\tau_{{D}}$ in $ms$, Trace lengths, Traces used]
Index: []

    We confirmed that the correlation did not fail on none of the test traces (no NaN values).

  2. Now, first plot the trace lengths. 
    x = 'Trace lengths'

g = sns.FacetGrid(data=corr_out,
                  col='Simulated $D$',
                  col_wrap=2,
                  sharex=True,
                  aspect=1.5,
                  height=5,
                  legend_out=True)
g.map_dataframe(sns.boxplot,
      x=x,
      y='Traces used',
      hue='Simulated $D_{{clust}}$',
      palette='colorblind',
      showfliers=False)
g.add_legend(title='Simulated $D_{{clust}}$')
g.map_dataframe(sns.stripplot,
      x=x,
      y='Traces used',
      hue='Simulated $D_{{clust}}$',
      dodge=True,
      palette=sns.color_palette(['0.3']),
      size=4,
      jitter=0.2)
g.set_xlabels(x)
g.fig.patch.set_facecolor('white')
g.tight_layout()
plt.show()

    Check out the trace lengths. The reduction seems similar over all simulated data. This speaks to a good generalization. Compared to exp-210204-unet, there is especially an improvement in the Simulated D of 0.1, and 0.4 to 50 and Simulated \(D_{clust}\) of 0.01. 2021-12-15_correlations_trace-lengths.png

  3. Now let's take a look at the diffrates and transit times. Since the distribution follows a log normal, use a log scale. 
    # '$D$ in $\\frac{{\mu m^2}}{{s}}$'
# '$\\tau_{{D}}$ in $ms$'
x = '$\\tau_{{D}}$ in $ms$'

g = sns.FacetGrid(data=corr_out,
                  col='Simulated $D$',
                  col_wrap=2,
                  sharex=False,  # False for x=D or x=tau, True for x=Trace lengths
                  aspect=1.5,
                  height=5,
                  legend_out=True)
g.map_dataframe(sns.boxplot,
      x=x,
      y='Traces used',
      hue='Simulated $D_{{clust}}$',
      palette='colorblind',
      showfliers=False).set(xscale = 'log')
g.add_legend(title='Simulated $D_{{clust}}$')
g.map_dataframe(sns.stripplot,
      x=x,
      y='Traces used',
      hue='Simulated $D_{{clust}}$',
      dodge=True,
      palette=sns.color_palette(['0.3']),
      size=4,
      jitter=0.2).set(xscale = 'log')
g.set_xlabels(x)
g.fig.patch.set_facecolor('white')
g.tight_layout()
plt.show()

    D: 2021-12-15_correlations_diffrates.png

    We see again a strong improvement in \(D_{clust} = 0.01\), this time with all simulated \(D\). We also see that there are occasional outliers in the prediction group (e.g. \(D = 50\) and \(D_{clust} = 0.01\)), but the statistics are in general okay now.

    Still things to look out for: for \(D=50\) we see a slight distortion to lower transit times / faster diffrates, in both reduction by label and prediction. Here seems to lie a boundary of the correction method.

  4. Lastly, let's take a look at a scatterplot of transit times vs trace lengths using a subsample to avoid overplotting. 
    corr_scatter = corr_out[corr_out['Traces used'].isin(['corrected by labels (control)', 'corrected by prediction'])].sample(1000)
corr_scatter
      Simulated \(D\) Simulated \(D_{{clust}}\) nmol \(D\) in \(\frac{{\mu m^2}}{{s}}\) \(\tau_{{D}}\) in \(ms\) Trace lengths Traces used
    9099 0.069 1.00 679.0 0.053497 210.684366 15232 corrected by prediction
    9032 0.069 1.00 679.0 0.065549 171.949847 15583 corrected by prediction
    10678 0.069 0.10 897.0 0.023812 473.327986 15480 corrected by prediction
    9556 3.000 0.01 1785.0 3.321410 3.393455 13251 corrected by prediction
    11319 0.100 0.10 1156.0 0.075619 149.050995 14235 corrected by prediction
    … … … … … … … …
    6002 0.069 1.00 679.0 0.060130 187.443486 14908 corrected by labels (control)
    9256 0.400 0.10 1939.0 0.398673 28.271429 15496 corrected by prediction
    9094 0.069 1.00 679.0 0.016429 686.039288 14580 corrected by prediction
    11329 0.100 0.10 1156.0 0.165504 68.101517 9384 corrected by prediction
    11012 1.000 0.01 1357.0 1.491715 7.555768 4462 corrected by prediction

    1000 rows × 7 columns 

    # '$D$ in $\\frac{{\mu m^2}}{{s}}$'
# '$\\tau_{{D}}$ in $ms$'
x = '$D$ in $\\frac{{\mu m^2}}{{s}}$'
g = sns.relplot(
          data=corr_scatter,
          x=x, y="Trace lengths",
          row="Traces used",
          hue="Simulated $D$",
          style="Simulated $D_{{clust}}$",
          kind="scatter",
          aspect=1.5,
          palette='colorblind').set(xscale='log')
g.set_xlabels(x)
g.tight_layout()
g.fig.patch.set_facecolor('white')
plt.show()

    We see that the distribution for \(D\) starts to spread out after rougly 10% of a trace has been deleted. The traces which had the most artifacts in them and thus were corrected most, were with a \(D_{clust}\) of 0.01. 2021-12-15_correlations_tracelengths-vs-diffrate-scatterplot.png

2.1.8 Learnings from run 1 and 2

  • The f1 metric does not work
  • I need to fix the fitting algorithm - as can be seen in <exp-…> the binning to 100um is not sufficient to correlate the plateau, thus we fit only the tail. This could be one reason why we get an average / median transit time of 0.5ms for uncorrupted traces, even though Pablo fitted 0.225ms in FOCUSpoint.
  • Boundary of correction method seems to be D=50, here correction by label and correction by prediction reduce the transit times too much, as in the achieved transit times are too low / diffrates are too fast.