AMD Ryzen AIチュートリアルAdvanced Quark Quantize

AMD Ryzen AIソフトウェアって？

AMD Ryzen AIチュートリアルAdvanced Quark Quantize

AMD Ryzen AIチュートリアルAdvanced Quark Quantize

　AMD Ryzen AIソフトウェアとNPUドライバをインストールした自身初購入のミニパソコンRyzen 7 8845HS搭載AOOSTAR GEM12 Pro MAX/Windows 11 Proにamd / RyzenAI-SWをgit cloneしてAI学習するAdvanced Quark Quantize編。

　Advanced Quark QuantizeもQuark Quantizationをルートディレクトリとしたものですが、文量が多くなる、わかりにくくなるかなとページを分けることにしました。

• advanced_quark_quantizationの概要
• advanced_quark_quantizationを実行する前に
• 仮想環境作成と必須パッケージインストール
• Ryzen AI SoftwareチュートリアルAdvanced Quark Quantization [MobileNetV2モデル]
• Ryzen AI SoftwareチュートリアルAdvanced Quark Quantization [ResNet50モデル]
• 終わったらconda deactivateを忘れずに

advanced_quark_quantizationの概要

　quark_quantization同様、詳細は、検索しても、なかなか見当たらず、quark_quantizationルート直下のdocディレクトリのadvanced_quark_quant_readme.mdを参照。

　より複雑なケースでは、精度を保つのが難しくなることがあり、AMD Quarkには、こうしたケースでも精度維持・向上させるFast Fine Tune(微調整)とCross Layer Equalization(クロス層平準化？)という2つのオプションがあり、CPUについては、Float32とINT8、NPUについては、INT8でオリジナルとこれら2つの精度比較を行うのが、このチュートリアル。

　MobileNetV2モデルとResNet50モデルで検証。

advanced_quark_quantizationを実行する前に

　今日時点、README.mdのこのリポジトリからというリンクには、[mobilenetv2-12.onnx]とありつつ、advance_quark_quantize.pyのinput部が[mobilenetv2]となっており、エラーとなったので、これを修正、実行時の引数の--model_inputの値も同じだったので[mobilenetv2-12]として実行する必要がありました。

仮想環境作成と必須パッケージインストール

　何らかの理由で当該ディレクトリで、まだ仮想環境をアクティベートしてないければ、また、requirements.txtを使ってパッケージ群をインストールしていなければ、

　quicktest時と同様に仮想環境を作って、仮想環境を有効にしてから始めます。

　ryzen-ai-1.4.0は、Ryzen AIソフトウェアのインストール中に指定時、デフォルトのconda仮想環境名、もしくは、それをベースに変更したconda仮想環境名です。

　続いて当該プロジェクトで必要なパッケージのインストール。

Ryzen AI SoftwareチュートリアルAdvanced Quark Quantization [MobileNetV2モデル]

　更に訓練前のResNet50モデルを取得するためにmodels/フォルダに移動してpython download_MobileNetV2.pyを実行します。

　次の作業のためにquark_quantizationフォルダに戻っておきます。

　次にHugging FaceのILSVRC/imagenet-1kからval_images.tar.gzファイルをダウンロードします。

　尚、今日時点では、この時、Hugging Faceへの会員登録とログイン後に表示される規約をAccept(許諾)する必要がありました。

　会員登録については、当該ページの少し下に[Log in] or [Sign up]というボタンがあり、登録なら後者をクリックします。

　登録後、同じ位置あたりに規約と[Accept]といったボタンがあると思うのでクリックしてから、その下にあるリストからダウンロードします。

　プロジェクトルートのquark_quantization上でHugging Faceからダウンロードしたval_images.tar.gzをこのようにval_dataフォルダに展開し、prepare_data.pyを実行、val_dataを処理した結果をcalib_dataというフォルダに保存します。

　ちなみに自身は、これ以前に現時点で、まだ終わっていないyolov8やtorchvision_inferenceもいじっていて後者でval_dataにあたるものをcaib_dataとしてダウンロードしてしまい、フォルダ分けしてねーじゃねーか！エラーを食らっており、ここでダウンロードするまでもなく、そのフォルダをmv、val_dataフォルダとして保存したというのは内緒。

　ということは、もしかして、ここで作ったcalib_dataがtorchvision_inferenceで使えるのか？と思ってみたりもしますが、いくつかフォルダを覗いてみてもJpeg画像1つしか入ってなかったりで、はて違うかな？と思ってみたり。

　というか、そもそも、torchvision_inferenceでフォルダに分類されてないよって言われたスクリプトは、classification.pyなんですが、それこそが、分類してくれるものじゃないの？だって、まさか事前準備したデータセットが分類されてるか否かをチェックするスクリプトじゃないもんね？と思ってみたり。

[2025/04/04]　やはり、このcalib_dataを使ってみたら、torchvision_inferenceも上手く実行できました。

　さておき、高度な量子化におけるMobileNetV2、INT8で精度に関するオプションなしの量子化advanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnxを実行します。

　続いて高度な量子化におけるMobileNetV2、INT8で--fast_finetuneオプション付きの量子化advanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnx --fast_finetuneを実行。

　最後に高度な量子化におけるMobileNetV2、INT8での --cross_layer_equalizationオプション付きのadvanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnx --cross_layer_equalizationを実行。

　結果となる部分だけ抜粋、と思ったら、ちょっと離れすぎて見づらいので...

　こんな感じの方がよいかな。

　精度なので小数点値は、1を100としたパーセンテージということでしょう。

　CPU Float32は全て同じ、これに対し、INT8を比較、CPUとNPUは、何れも精度に関するオプションなしに比べてCross Layer Equalization、Fast Fine Tuneの順に精度が上ってますね。

　INT8より精度が高いと思われるFloat32モデル(CPU)と比べても、INT8なCPU・NPU共にCross Layer Equalizationは少し精度が落ちる程度、Fast Fine Tuneは、精度がより高くなっていますね。

　って、INT8の(NPU)と書いてない方は、CPUでよいんだよね？同じっておかしくない？そういうもの？

　ちなみに今回もタスクマネージャは開きつつ、実行したものの、ながら作業をしていたりで、よく見ていませんでした。

Ryzen AI SoftwareチュートリアルAdvanced Quark Quantization [ResNet50モデル]

　次のエラーに遭遇、原因究明中。

[Unable to allocate 2.99 GiB for an array with shape (802816000,) and data type float32]

　Hawk Pointだとダメなのかな？現在、調査中...。

　というかメモリ不足なんでしょうが、エラーとなったサイズ自体は、64ビットで不足するほどではなく、タスクマネージャを見ても余力はありそう、WindowsもPythonも64ビットではある次のエラーに遭遇、原因究明中。

　ただ、Pythonインタプリタを起動してみると1行めに[on win32]とかいう出力もあり、32ビットAPIとかに起因するものなのか、多次元配列とか使うと巨大なサイズになってしまうということなのか、スクリプトの修正が必要なのか、物理メモリ不足なのかとか、よくわからないので。

終わったらconda deactivateを忘れずに

　condaの仮想環境での作業が一通り終わったら、conda deactivate(activateの反対語deactivate)しましょう。

　これを忘れると、たいてい痛い目にあうので要注意。

　conda deactivateしたら、都度、conda info -eで[*]のついた仮想環境がないことを確認しておくと安心です。

　なぜなら、一見、作業ディレクトリパス先頭から(ryzen-ai-1.4.0)といった仮想環境の表記が消えてもconda info -eで確認すると、まだ[*]が付いているなんてことがあったり、なぜか、代わりに(base)という仮想環境に入ってしまっていたり、知らずにAnacondaコマンドプロンプトを閉じて戻っても同様だったりすることもあったりするので。

　仮想環境baseになってしまう場合、ディレクトリパス先頭に表示されるとは言え、本来の仮想環境については、deactivateコマンドを実行した確信が手伝うのか、意外と気づかないことも多く...。

　何れにせよ、condaの仮想環境にいることを、知らずに、そのまま作業し続けると何かとハマりやすいので要注意。

仮想環境作成と必須パッケージインストール

(ryzen-ai-1.4.0) path\to\quark_quantization> python -m virtualenv .venv

(ryzen-ai-1.4.0) path\to\quark_quantization> .\.venv\Scripts\activate

　ページを分けた関係で改めて書くと、まずは、quicktest時と同様に仮想環境を作って、仮想環境を有効にしてから始めます。

　ここでは、condaではなく、pipを使いますが、pip install virtualenvは既に済んでいるものとします。

(ryzen-ai-1.4.0) path\to\quark_quantization> pip install -r requirements.txt

　続いて当該プロジェクトで必要なパッケージのインストール。

Ryzen AI SoftwareチュートリアルAdvanced Quark Quantization [MobileNetV2モデル]

(ryzen-ai-1.4.0) path\to\quark_quantization> cd model

(ryzen-ai-1.4.0) path\to\quark_quantization\model> python download_ResNet.py

(ryzen-ai-1.4.0) path\to\quark_quantization\model> cd ..

(ryzen-ai-1.4.0) path\to\quark_quantization>

　更に訓練前のResNet50モデルを取得するためにmodel/フォルダに移動してpython download_ResNet.pyを実行します。

　次の作業のためにquark_quantizationフォルダに戻っておきます。

　次にHugging FaceのILSVRC/imagenet-1kからval_images.tar.gzファイルをダウンロードします。

　尚、今日時点では、この時、Hugging Faceへの会員登録とログイン後に表示される規約をAccept(許諾)する必要がありました。

　会員登録については、当該ページの少し下に[Log in] or [Sign up]というボタンがあり、登録なら後者をクリックします。

　登録後、同じ位置あたりに規約と[Accept]といったボタンがあると思うのでクリックしてから、その下にあるリストからダウンロードします。

(ryzen-ai-1.4.0) path\to\quark_quantization> mkdir val_data && tar -xzf val_images.tar.gz -C val_data

(ryzen-ai-1.4.0) path\to\quark_quantization> python prepare_data.py val_data calib_data

　プロジェクトルートのquark_quantization上でHugging Faceからダウンロードしたval_images.tar.gzをこのようにval_dataフォルダに展開し、prepare_data.pyを実行、val_dataを処理した結果をcalib_dataというフォルダに保存します。

　ちなみに自身は、これ以前に現時点で、まだ終わっていないyolov8やtorchvision_inferenceもいじっていて後者でval_dataにあたるものをcaib_dataとしてダウンロードしてしまい、フォルダ分けしてねーじゃねーか！エラーを食らっており、ここでダウンロードするまでもなく、そのフォルダをmv、val_dataフォルダとして保存したというのは内緒。

　ということは、もしかして、ここで作ったcalib_dataがtorchvision_inferenceで使えるのか？と思ってみたりもしますが、いくつかフォルダを覗いてみてもJpeg画像1つしか入ってなかったりで、はて違うかな？と思ってみたり。

　というか、そもそも、torchvision_inferenceでフォルダに分類されてないよって言われたスクリプトは、classification.pyなんですが、それこそが、分類してくれるものじゃないの？だって、まさか事前準備したデータセットが分類されてるか否かをチェックするスクリプトじゃないもんね？と思ってみたり。

(venv) PS path\to\quark_quantization> python advanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnx

 

[QUARK-INFO]: Checking custom ops library ...

 

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-INFO]: Start compiling CPU version of custom ops library.

path\to\quark_quantization\venv\Lib\site-packages\torch\utils\cpp_extension.py:414: UserWarning: Error checking compiler version for cl: [WinError 2] 指定されたファイルが見つかりません。

 warnings.warn(f'Error checking compiler version for {compiler}: {error}')

INFO: Could not find files for the given pattern(s).

 

[QUARK-ERROR]: CPU version of custom ops library compilation failed:Command '['where', 'cl']' returned non-zero exit status 1.

 

[QUARK-WARNING]: Custom ops library compilation failed: CPU version of custom ops library compilation failed:Command '['where', 'cl']' returned non-zero exit status 1..

 

[QUARK-INFO]: Checked custom ops library.

The configuration of the quantization is Config(global_quant_config=QuantizationConfig(calibrate_method=<PowerOfTwoMethod.MinMSE: 1>, quant_format=<QuantFormat.QDQ: 1>, activation_type=<QuantType.QUInt8: 1>, weight_type=<QuantType.QInt8: 0>, input_nodes=[], output_nodes=[], op_types_to_quantize=[], nodes_to_quantize=[], extra_op_types_to_quantize=[], nodes_to_exclude=[], subgraphs_to_exclude=[], specific_tensor_precision=False, execution_providers=['CPUExecutionProvider'], per_channel=False, reduce_range=False, optimize_model=True, use_dynamic_quant=False, use_external_data_format=False, convert_fp16_to_fp32=False, convert_nchw_to_nhwc=False, include_sq=False, include_rotation=False, include_cle=False, include_auto_mp=False, include_fast_ft=False, enable_npu_cnn=True, enable_npu_transformer=False, debug_mode=False, print_summary=True, ignore_warnings=True, log_severity_level=1, extra_options={'ActivationSymmetric': True}))

[QUARK_INFO]: Time information:

2025-04-04 22:45:37.150566

[QUARK_INFO]: OS and CPU information:

                    system --- Windows

                     node --- gem12promax

                    release --- 11

                    version --- 10.0.26100

                    machine --- AMD64

                   processor --- AMD64 Family 25 Model 117 Stepping 2, AuthenticAMD

[QUARK_INFO]: Tools version information:

                    python --- 3.12.0

                     onnx --- 1.17.0

                  onnxruntime --- 1.20.1

                  quark.onnx --- 0.8+103c340fe2

[QUARK_INFO]: Quantized Configuration information:

                  model_input --- models/mobilenetv2-12.onnx

                 model_output --- models/mobilenetv2_quant.onnx

            calibration_data_reader --- <utils.ImageDataReader object at 0x00000185EE260860>

             calibration_data_path --- None

                 quant_format --- QDQ

                  input_nodes --- []

                 output_nodes --- []

             op_types_to_quantize --- []

          extra_op_types_to_quantize --- []

                  per_channel --- False

                 reduce_range --- False

                activation_type --- QUInt8

                  weight_type --- QInt8

               nodes_to_quantize --- []

               nodes_to_exclude --- []

             subgraphs_to_exclude --- []

                optimize_model --- True

           use_external_data_format --- False

               calibrate_method --- PowerOfTwoMethod.MinMSE

              execution_providers --- ['CPUExecutionProvider']

                enable_npu_cnn --- True

            enable_npu_transformer --- False

           specific_tensor_precision --- False

                  debug_mode --- False

             convert_fp16_to_fp32 --- False

             convert_nchw_to_nhwc --- False

                  include_cle --- False

                  include_sq --- False

               include_rotation --- False

                include_fast_ft --- False

                 extra_options --- {'ActivationSymmetric': True}

 

[QUARK-INFO]: The input ONNX model models/mobilenetv2-12.onnx can create InferenceSession successfully

 

[QUARK-INFO]: Obtained calibration data with 1000 iters

 

[QUARK-INFO]: Removed initializers from input

 

[QUARK-INFO]: Simplified model sucessfully

 

[QUARK-INFO]: Duplicate the shared initializers in the model for separate quantization use across different nodes!

 

[QUARK-INFO]: Loading model...

 

[QUARK-INFO]: The input ONNX model C:/Users/reg/AppData/Local/Temp/vai.cpinit.w15j7yng/model_cpinit.onnx can run inference successfully

 

[QUARK-INFO]: optimize the model for better hardware compatibility.

 

[QUARK-WARNING]: The opset version is 17 < 20. Skipping fusing Gelu.

 

[QUARK-INFO]: Start calibration...

 

[QUARK-INFO]: Start collecting data, runtime depends on your model size and the number of calibration dataset.

 

[QUARK-INFO]: Finding optimal threshold for each tensor using PowerOfTwoMethod.MinMSE algorithm ...

 

[QUARK-INFO]: Use all calibration data to calculate min mse

Computing range: 100%|█████████████████████████████████████████████████| 101/101 [15:31<00:00, 9.23s/tensor]

 

[QUARK-INFO]: Finished the calibration of PowerOfTwoMethod.MinMSE which costs 1022.2s

 

[QUARK-INFO]: Remove QuantizeLinear & DequantizeLinear on certain operations(such as conv-relu).

 

[QUARK-INFO]: Rescale GlobalAveragePool /GlobalAveragePool with factor 1.0048828125 to simulate DPU behavior.

 

[QUARK-INFO]: Adjust the quantize info to meet the compiler constraints

 

[QUARK-INFO]: Input pos of pooling layer /GlobalAveragePool is 4. Output pos of pooling layer /GlobalAveragePool is 5.Modify opos from 5 to 4.

 

[QUARK-INFO]: Adjust the quantize info to meet the compiler constraints

The operation types and their corresponding quantities of the input float model is shown in the table below.

┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓

┃ Op Type       ┃ Float Model          ┃

┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩

│ Conv         │ 52              │

│ Constant       │ 70              │

│ Clip         │ 35              │

│ Add         │ 10              │

│ GlobalAveragePool  │ 1               │

│ Flatten       │ 1               │

│ Gemm         │ 1               │

├──────────────────────┼───────────────────────────────┤

│ Quantized model path │ models/mobilenetv2_quant.onnx │

└──────────────────────┴───────────────────────────────┘

The quantized information for all operation types is shown in the table below.

The discrepancy between the operation types in the quantized model and the float model is due to the application of graph optimization.

┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓

┃ Op Type      ┃ Activation ┃ Weights ┃ Bias   ┃

┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩

│ Conv       │ UINT8(52) │ INT8(52) │ INT8(52) │

│ Add        │ UINT8(10) │     │     │

│ GlobalAveragePool │ UINT8(1)  │     │     │

│ Flatten      │ UINT8(1)  │     │     │

│ Gemm       │ UINT8(1)  │ INT8(1) │ INT8(1) │

└───────────────────┴────────────┴──────────┴──────────┘

Model Size:

Float32 model size: 13.34 MB

Int8 quantized model size: 3.45 MB

Model Accuracy:

Evaluating: 100%|███████████████████████████████████████████████████████| 1000/1000 [00:07<00:00, 127.30it/s]

Float32 model accuracy: Top1 0.713, Top5 0.906

Evaluating: 100%|████████████████████████████████████████████████████████| 1000/1000 [00:22<00:00, 45.40it/s]

Int8 quantized model accuracy: Top1 0.642, Top5 0.869

path\to\quark_quantization

Evaluating: 100%|████████████████████████████████████████████████████████| 1000/1000 [00:21<00:00, 46.24it/s]

Int8 quantized model accuracy (NPU): Top1 0.642, Top5 0.869

(venv) PS path\to\quark_quantization>

　さておき、高度な量子化におけるMobileNetV2、INT8でのオプションなしの量子化advanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnxを実行します。

(venv) PS path\to\quark_quantization> python advanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnx --fast_finetune

 

[QUARK-INFO]: Checking custom ops library ...

 

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-INFO]: Start compiling CPU version of custom ops library.

path\to\quark_quantization\venv\Lib\site-packages\torch\utils\cpp_extension.py:414: UserWarning: Error checking compiler version for cl: [WinError 2] 指定されたファイルが見つかりません。

 warnings.warn(f'Error checking compiler version for {compiler}: {error}')

INFO: Could not find files for the given pattern(s).

 

[QUARK-ERROR]: CPU version of custom ops library compilation failed:Command '['where', 'cl']' returned non-zero exit status 1.

 

[QUARK-WARNING]: Custom ops library compilation failed: CPU version of custom ops library compilation failed:Command '['where', 'cl']' returned non-zero exit status 1..

 

[QUARK-INFO]: Checked custom ops library.

The configuration of the quantization is Config(global_quant_config=QuantizationConfig(calibrate_method=<PowerOfTwoMethod.MinMSE: 1>, quant_format=<QuantFormat.QDQ: 1>, activation_type=<QuantType.QUInt8: 1>, weight_type=<QuantType.QInt8: 0>, input_nodes=[], output_nodes=[], op_types_to_quantize=[], nodes_to_quantize=[], extra_op_types_to_quantize=[], nodes_to_exclude=[], subgraphs_to_exclude=[], specific_tensor_precision=False, execution_providers=['CPUExecutionProvider'], per_channel=False, reduce_range=False, optimize_model=True, use_dynamic_quant=False, use_external_data_format=False, convert_fp16_to_fp32=False, convert_nchw_to_nhwc=False, include_sq=False, include_rotation=False, include_cle=False, include_auto_mp=False, include_fast_ft=True, enable_npu_cnn=True, enable_npu_transformer=False, debug_mode=False, print_summary=True, ignore_warnings=True, log_severity_level=1, extra_options={'ActivationSymmetric': True}))

[QUARK_INFO]: Time information:

2025-04-05 08:51:41.563338

[QUARK_INFO]: OS and CPU information:

                    system --- Windows

                     node --- gem12promax

                    release --- 11

                    version --- 10.0.26100

                    machine --- AMD64

                   processor --- AMD64 Family 25 Model 117 Stepping 2, AuthenticAMD

[QUARK_INFO]: Tools version information:

                    python --- 3.12.0

                     onnx --- 1.17.0

                  onnxruntime --- 1.20.1

                  quark.onnx --- 0.8+103c340fe2

[QUARK_INFO]: Quantized Configuration information:

                  model_input --- models/mobilenetv2-12.onnx

                 model_output --- models/mobilenetv2_quant.onnx

            calibration_data_reader --- <utils.ImageDataReader object at 0x000001EC3C69AD50>

             calibration_data_path --- None

                 quant_format --- QDQ

                  input_nodes --- []

                 output_nodes --- []

             op_types_to_quantize --- []

          extra_op_types_to_quantize --- []

                  per_channel --- False

                 reduce_range --- False

                activation_type --- QUInt8

                  weight_type --- QInt8

               nodes_to_quantize --- []

               nodes_to_exclude --- []

             subgraphs_to_exclude --- []

                optimize_model --- True

           use_external_data_format --- False

               calibrate_method --- PowerOfTwoMethod.MinMSE

              execution_providers --- ['CPUExecutionProvider']

                enable_npu_cnn --- True

            enable_npu_transformer --- False

           specific_tensor_precision --- False

                  debug_mode --- False

             convert_fp16_to_fp32 --- False

             convert_nchw_to_nhwc --- False

                  include_cle --- False

                  include_sq --- False

               include_rotation --- False

                include_fast_ft --- True

                 extra_options --- {'ActivationSymmetric': True}

 

[QUARK-INFO]: The input ONNX model models/mobilenetv2-12.onnx can create InferenceSession successfully

 

[QUARK-INFO]: Obtained calibration data with 1000 iters

 

[QUARK-INFO]: Removed initializers from input

 

[QUARK-INFO]: Simplified model sucessfully

 

[QUARK-INFO]: Duplicate the shared initializers in the model for separate quantization use across different nodes!

 

[QUARK-INFO]: Loading model...

 

[QUARK-INFO]: The input ONNX model C:/Users/reg/AppData/Local/Temp/vai.cpinit.41jr_qk2/model_cpinit.onnx can run inference successfully

 

[QUARK-INFO]: optimize the model for better hardware compatibility.

 

[QUARK-WARNING]: The opset version is 17 < 20. Skipping fusing Gelu.

 

[QUARK-INFO]: Start calibration...

 

[QUARK-INFO]: Start collecting data, runtime depends on your model size and the number of calibration dataset.

 

[QUARK-INFO]: Finding optimal threshold for each tensor using PowerOfTwoMethod.MinMSE algorithm ...

 

[QUARK-INFO]: Use all calibration data to calculate min mse

Computing range: 100%|███████████████████████████████████████████████████████████| 101/101 [15:34<00:00, 9.25s/tensor]

 

[QUARK-INFO]: Finished the calibration of PowerOfTwoMethod.MinMSE which costs 1040.6s

 

[QUARK-INFO]: Remove QuantizeLinear & DequantizeLinear on certain operations(such as conv-relu).

 

[QUARK-INFO]: Rescale GlobalAveragePool /GlobalAveragePool with factor 1.0048828125 to simulate DPU behavior.

 

[QUARK-INFO]: Adjust the quantize info to meet the compiler constraints

 

[QUARK-INFO]: Input pos of pooling layer /GlobalAveragePool is 4. Output pos of pooling layer /GlobalAveragePool is 5.Modify opos from 5 to 4.

 

[QUARK-INFO]: Adjust the quantize info to meet the compiler constraints

 

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-INFO]: Start running fast finetune with seed 1743811744 ...

 0%|                                              | 0/53 [00:00<?, ?it/s]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (input)->(/features/features.0/features.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=0.027316 (Recons loss=0.027316, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=0.031669 (Recons loss=0.031669, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=6.894515 (Recons loss=0.022314, Rounding loss=6.872201)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=3.756062 (Recons loss=0.026928, Rounding loss=3.729134)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=3.307524 (Recons loss=0.043902, Rounding loss=3.263623)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=2.730858 (Recons loss=0.025476, Rounding loss=2.705381)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=2.239303 (Recons loss=0.028903, Rounding loss=2.210400)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=1.225353 (Recons loss=0.028312, Rounding loss=1.197041)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=0.326279 (Recons loss=0.042230, Rounding loss=0.284049)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=0.068286 (Recons loss=0.068286, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=0.064702 (Recons loss=0.064702, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (input)->(/features/features.0/features.0.2/Clip_output_0) recons metrics was optimized from 0.011162 to 0.001879 (diff=-0.009283)

 2%|█▌                                         | 1/53 [00:40<35:22, 40.81s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.0/features.0.2/Clip_output_0)->(/features/features.1/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=0.403004 (Recons loss=0.403004, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=0.357111 (Recons loss=0.357111, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=2.598006 (Recons loss=0.367849, Rounding loss=2.230158)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=1.658811 (Recons loss=0.349553, Rounding loss=1.309259)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=1.516412 (Recons loss=0.366669, Rounding loss=1.149743)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=1.391208 (Recons loss=0.379066, Rounding loss=1.012142)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=1.235395 (Recons loss=0.389620, Rounding loss=0.845776)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=1.026012 (Recons loss=0.387075, Rounding loss=0.638937)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=0.597411 (Recons loss=0.442416, Rounding loss=0.154994)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=0.427725 (Recons loss=0.401189, Rounding loss=0.026535)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=0.440987 (Recons loss=0.440987, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.0/features.0.2/Clip_output_0)->(/features/features.1/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.022396 to 0.014502 (diff=-0.007894)

 4%|███▏                                        | 2/53 [01:07<27:27, 32.30s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.1/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.1/conv/conv.1/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=5.780297 (Recons loss=5.780297, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=4.520518 (Recons loss=4.520518, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=7.195956 (Recons loss=4.192791, Rounding loss=3.003165)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=5.812351 (Recons loss=3.904629, Rounding loss=1.907722)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=5.630322 (Recons loss=3.938168, Rounding loss=1.692153)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=5.972135 (Recons loss=4.443610, Rounding loss=1.528525)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=5.422660 (Recons loss=4.028100, Rounding loss=1.394560)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=4.941269 (Recons loss=3.776690, Rounding loss=1.164578)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=4.110936 (Recons loss=3.385652, Rounding loss=0.725284)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=3.919687 (Recons loss=3.721004, Rounding loss=0.198683)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=4.021281 (Recons loss=3.972682, Rounding loss=0.048600)

 

[QUARK-INFO]: Module (/features/features.1/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.1/conv/conv.1/Conv_output_0) recons metrics was optimized from 0.409014 to 0.276108 (diff=-0.132907)

 6%|████▋                                       | 3/53 [01:28<22:39, 27.20s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.1/conv/conv.1/Conv_output_0)->(/features/features.2/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=6.418439 (Recons loss=6.418439, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=4.280893 (Recons loss=4.280893, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=16.915007 (Recons loss=4.221501, Rounding loss=12.693505)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=13.561399 (Recons loss=5.083219, Rounding loss=8.478180)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=12.328688 (Recons loss=4.200441, Rounding loss=8.128247)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=12.796868 (Recons loss=5.164981, Rounding loss=7.631887)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=11.244919 (Recons loss=4.337056, Rounding loss=6.907864)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=10.510124 (Recons loss=5.242612, Rounding loss=5.267512)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=8.281898 (Recons loss=5.214523, Rounding loss=3.067375)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=7.569227 (Recons loss=6.732896, Rounding loss=0.836331)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=6.352596 (Recons loss=5.983551, Rounding loss=0.369046)

 

[QUARK-INFO]: Module (/features/features.1/conv/conv.1/Conv_output_0)->(/features/features.2/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.080139 to 0.071235 (diff=-0.008904)

 8%|██████▎                                      | 4/53 [02:26<32:18, 39.56s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.2/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.2/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=4.238964 (Recons loss=4.238964, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=2.757794 (Recons loss=2.757794, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=6.584691 (Recons loss=3.315610, Rounding loss=3.269081)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=3.537403 (Recons loss=1.919521, Rounding loss=1.617881)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=3.802451 (Recons loss=2.532621, Rounding loss=1.269830)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=2.597032 (Recons loss=1.597098, Rounding loss=0.999934)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=4.911505 (Recons loss=4.261745, Rounding loss=0.649759)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=3.683266 (Recons loss=3.353741, Rounding loss=0.329525)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=2.509097 (Recons loss=2.479211, Rounding loss=0.029886)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=2.358015 (Recons loss=2.358015, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=2.983615 (Recons loss=2.983615, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.2/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.2/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.034339 to 0.029851 (diff=-0.004488)

 9%|███████▊                                      | 5/53 [04:14<51:15, 64.07s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.2/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.2/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=27.280249 (Recons loss=27.280249, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=23.887402 (Recons loss=23.887402, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=41.107372 (Recons loss=33.173447, Rounding loss=7.933926)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=22.414654 (Recons loss=18.921556, Rounding loss=3.493097)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=27.045141 (Recons loss=24.809765, Rounding loss=2.235376)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=28.554375 (Recons loss=27.107592, Rounding loss=1.446782)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=17.531712 (Recons loss=16.674355, Rounding loss=0.857356)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=17.334557 (Recons loss=16.903734, Rounding loss=0.430822)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=27.540363 (Recons loss=27.362476, Rounding loss=0.177887)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=24.340355 (Recons loss=24.311052, Rounding loss=0.029302)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=15.564527 (Recons loss=15.552215, Rounding loss=0.012312)

 

[QUARK-INFO]: Module (/features/features.2/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.2/conv/conv.2/Conv_output_0) recons metrics was optimized from 1.331489 to 1.048152 (diff=-0.283338)

11%|█████████▍                                     | 6/53 [04:35<38:53, 49.66s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.2/conv/conv.2/Conv_output_0)->(/features/features.3/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=4.171536 (Recons loss=4.171536, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=4.456164 (Recons loss=4.456164, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=21.726088 (Recons loss=4.060321, Rounding loss=17.665766)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=11.615293 (Recons loss=3.489615, Rounding loss=8.125677)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=11.208338 (Recons loss=4.647383, Rounding loss=6.560955)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=9.587914 (Recons loss=4.344686, Rounding loss=5.243228)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=6.437772 (Recons loss=3.209199, Rounding loss=3.228572)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=5.822534 (Recons loss=4.185576, Rounding loss=1.636957)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=4.685205 (Recons loss=4.570946, Rounding loss=0.114259)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=4.793823 (Recons loss=4.793823, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=5.730151 (Recons loss=5.730151, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.2/conv/conv.2/Conv_output_0)->(/features/features.3/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.030997 to 0.028928 (diff=-0.002069)

13%|██████████▉                                    | 7/53 [04:58<31:13, 40.73s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.3/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.3/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=3.875463 (Recons loss=3.875463, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=3.804551 (Recons loss=3.804551, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=14.025936 (Recons loss=4.369352, Rounding loss=9.656584)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=9.415411 (Recons loss=3.690789, Rounding loss=5.724622)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=8.983266 (Recons loss=4.160852, Rounding loss=4.822414)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=9.412084 (Recons loss=5.098297, Rounding loss=4.313787)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=7.246734 (Recons loss=3.558779, Rounding loss=3.687955)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=6.610565 (Recons loss=4.010542, Rounding loss=2.600022)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=4.444396 (Recons loss=3.642246, Rounding loss=0.802150)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=4.958652 (Recons loss=4.940800, Rounding loss=0.017853)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=4.451784 (Recons loss=4.451784, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.3/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.3/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.032985 to 0.029743 (diff=-0.003242)

15%|████████████▌                                   | 8/53 [05:32<28:55, 38.56s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.3/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.3/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=39.054161 (Recons loss=39.054161, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=30.835724 (Recons loss=30.835724, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=54.256760 (Recons loss=35.466393, Rounding loss=18.790367)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=51.649296 (Recons loss=42.465305, Rounding loss=9.183990)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=56.347385 (Recons loss=49.147602, Rounding loss=7.199783)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=34.968174 (Recons loss=29.732428, Rounding loss=5.235746)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=28.612974 (Recons loss=24.981333, Rounding loss=3.631641)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=31.253620 (Recons loss=29.061750, Rounding loss=2.191870)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=43.190300 (Recons loss=42.403580, Rounding loss=0.786722)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=41.510616 (Recons loss=41.506233, Rounding loss=0.004384)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=29.974470 (Recons loss=29.974470, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.3/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.3/conv/conv.2/Conv_output_0) recons metrics was optimized from 1.774277 to 1.527473 (diff=-0.246804)

17%|██████████████                                   | 9/53 [05:55<24:52, 33.92s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.3/Add_output_0)->(/features/features.4/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=10.676511 (Recons loss=10.676511, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=12.137675 (Recons loss=12.137675, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=27.707268 (Recons loss=9.542767, Rounding loss=18.164501)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=20.389076 (Recons loss=11.411422, Rounding loss=8.977655)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=16.910156 (Recons loss=10.084881, Rounding loss=6.825276)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=16.359367 (Recons loss=11.051199, Rounding loss=5.308168)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=15.610174 (Recons loss=11.831778, Rounding loss=3.778397)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=13.446202 (Recons loss=11.439424, Rounding loss=2.006778)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=11.844906 (Recons loss=11.511014, Rounding loss=0.333892)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=13.279461 (Recons loss=13.279461, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=10.973071 (Recons loss=10.973071, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.3/Add_output_0)->(/features/features.4/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.080716 to 0.078419 (diff=-0.002297)

19%|███████████████▍                                 | 10/53 [06:19<22:06, 30.85s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.4/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.4/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=3.769478 (Recons loss=3.769478, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=4.758662 (Recons loss=4.758662, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=10.645983 (Recons loss=3.763649, Rounding loss=6.882334)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=6.964004 (Recons loss=3.833433, Rounding loss=3.130571)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=7.753238 (Recons loss=5.590448, Rounding loss=2.162790)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=5.155459 (Recons loss=3.731607, Rounding loss=1.423852)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=5.053861 (Recons loss=4.118468, Rounding loss=0.935393)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=4.616582 (Recons loss=4.197560, Rounding loss=0.419022)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=3.235956 (Recons loss=3.186927, Rounding loss=0.049028)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=4.019559 (Recons loss=4.019559, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=3.521901 (Recons loss=3.521901, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.4/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.4/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.028600 to 0.027770 (diff=-0.000831)

21%|█████████████████                                 | 11/53 [06:46<20:39, 29.50s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.4/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.4/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=30.163631 (Recons loss=30.163631, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=25.797882 (Recons loss=25.797882, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=58.332104 (Recons loss=28.237329, Rounding loss=30.094774)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=41.700909 (Recons loss=25.730112, Rounding loss=15.970798)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=56.380173 (Recons loss=44.521034, Rounding loss=11.859140)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=35.556301 (Recons loss=27.544905, Rounding loss=8.011397)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=34.157600 (Recons loss=28.756443, Rounding loss=5.401158)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=31.999035 (Recons loss=28.712168, Rounding loss=3.286868)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=30.643976 (Recons loss=29.345860, Rounding loss=1.298117)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=32.407082 (Recons loss=32.159210, Rounding loss=0.247871)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=27.536104 (Recons loss=27.515789, Rounding loss=0.020316)

 

[QUARK-INFO]: Module (/features/features.4/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.4/conv/conv.2/Conv_output_0) recons metrics was optimized from 1.040781 to 0.955003 (diff=-0.085778)

23%|██████████████████▌                                | 12/53 [07:01<17:03, 24.97s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.4/conv/conv.2/Conv_output_0)->(/features/features.5/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=3.989035 (Recons loss=3.989035, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=4.599865 (Recons loss=4.599865, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=40.687527 (Recons loss=3.601120, Rounding loss=37.086407)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=17.877682 (Recons loss=4.386981, Rounding loss=13.490702)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=12.885027 (Recons loss=4.111316, Rounding loss=8.773711)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=9.158084 (Recons loss=3.470505, Rounding loss=5.687580)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=6.543597 (Recons loss=3.854599, Rounding loss=2.688998)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=4.709496 (Recons loss=3.915256, Rounding loss=0.794241)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=4.008229 (Recons loss=3.978231, Rounding loss=0.029998)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=6.939166 (Recons loss=6.939166, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=4.760609 (Recons loss=4.760609, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.4/conv/conv.2/Conv_output_0)->(/features/features.5/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.024369 to 0.023851 (diff=-0.000518)

25%|████████████████████                               | 13/53 [07:15<14:33, 21.84s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.5/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.5/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=2.422610 (Recons loss=2.422610, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=3.380972 (Recons loss=3.380972, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=17.445967 (Recons loss=2.861824, Rounding loss=14.584143)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=12.729231 (Recons loss=3.692941, Rounding loss=9.036290)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=10.634861 (Recons loss=2.537175, Rounding loss=8.097686)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=9.201725 (Recons loss=2.025705, Rounding loss=7.176020)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=8.582280 (Recons loss=2.347270, Rounding loss=6.235010)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=6.363726 (Recons loss=2.030438, Rounding loss=4.333288)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=5.829529 (Recons loss=3.713465, Rounding loss=2.116065)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=2.646202 (Recons loss=2.510929, Rounding loss=0.135273)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=3.649445 (Recons loss=3.626822, Rounding loss=0.022623)

 

[QUARK-INFO]: Module (/features/features.5/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.5/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.023879 to 0.015907 (diff=-0.007972)

26%|█████████████████████▋                              | 14/53 [07:34<13:31, 20.82s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.5/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.5/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=14.035404 (Recons loss=14.035404, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=15.966422 (Recons loss=15.966422, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=62.766380 (Recons loss=19.689487, Rounding loss=43.076893)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=41.809917 (Recons loss=18.759682, Rounding loss=23.050236)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=31.412266 (Recons loss=12.257195, Rounding loss=19.155071)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=26.862728 (Recons loss=10.826461, Rounding loss=16.036268)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=23.735683 (Recons loss=11.536995, Rounding loss=12.198689)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=25.340473 (Recons loss=17.616623, Rounding loss=7.723850)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=18.110220 (Recons loss=14.953921, Rounding loss=3.156298)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=19.479006 (Recons loss=18.838881, Rounding loss=0.640126)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=18.410423 (Recons loss=18.318350, Rounding loss=0.092074)

 

[QUARK-INFO]: Module (/features/features.5/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.5/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.574525 to 0.461363 (diff=-0.113162)

28%|███████████████████████▏                             | 15/53 [07:49<12:12, 19.28s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.5/Add_output_0)->(/features/features.6/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=5.028424 (Recons loss=5.028424, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=5.221858 (Recons loss=5.221858, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=43.557106 (Recons loss=4.353369, Rounding loss=39.203735)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=23.892254 (Recons loss=6.273320, Rounding loss=17.618935)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=16.110037 (Recons loss=4.864428, Rounding loss=11.245609)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=12.018555 (Recons loss=4.952679, Rounding loss=7.065876)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=8.926890 (Recons loss=4.618710, Rounding loss=4.308181)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=5.760035 (Recons loss=4.245467, Rounding loss=1.514567)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=5.135393 (Recons loss=5.059085, Rounding loss=0.076308)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=4.400031 (Recons loss=4.400031, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=6.153133 (Recons loss=6.153133, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.5/Add_output_0)->(/features/features.6/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.028243 to 0.027801 (diff=-0.000442)

30%|████████████████████████▊                             | 16/53 [08:04<11:04, 17.96s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.6/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.6/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=2.902440 (Recons loss=2.902440, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=2.762220 (Recons loss=2.762220, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=13.833485 (Recons loss=3.056588, Rounding loss=10.776896)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=7.730373 (Recons loss=2.511383, Rounding loss=5.218990)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=6.349752 (Recons loss=2.383225, Rounding loss=3.966527)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=5.941700 (Recons loss=2.859976, Rounding loss=3.081723)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=5.284777 (Recons loss=3.008861, Rounding loss=2.275916)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=4.138593 (Recons loss=2.970637, Rounding loss=1.167956)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=3.068650 (Recons loss=2.986800, Rounding loss=0.081850)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=3.821405 (Recons loss=3.821405, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=4.256993 (Recons loss=4.256993, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.6/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.6/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.017777 to 0.016495 (diff=-0.001282)

32%|██████████████████████████▎                            | 17/53 [08:23<10:56, 18.24s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.6/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.6/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=18.059015 (Recons loss=18.059015, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=15.112957 (Recons loss=15.112957, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=57.978867 (Recons loss=15.680140, Rounding loss=42.298725)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=42.433777 (Recons loss=23.507200, Rounding loss=18.926579)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=29.609459 (Recons loss=15.799362, Rounding loss=13.810098)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=25.279751 (Recons loss=15.430839, Rounding loss=9.848911)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=29.440886 (Recons loss=22.985188, Rounding loss=6.455698)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=21.137228 (Recons loss=18.180054, Rounding loss=2.957174)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=14.238379 (Recons loss=13.689580, Rounding loss=0.548800)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=25.186852 (Recons loss=25.176744, Rounding loss=0.010108)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=20.035650 (Recons loss=20.035650, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.6/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.6/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.571938 to 0.518789 (diff=-0.053149)

34%|███████████████████████████▊                           | 18/53 [08:40<10:20, 17.73s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.6/Add_output_0)->(/features/features.7/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=7.443232 (Recons loss=7.443232, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=8.629070 (Recons loss=8.629070, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=48.724735 (Recons loss=6.445122, Rounding loss=42.279613)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=30.395958 (Recons loss=10.206188, Rounding loss=20.189770)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=22.033459 (Recons loss=7.211583, Rounding loss=14.821876)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=16.365110 (Recons loss=5.427557, Rounding loss=10.937553)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=14.070286 (Recons loss=6.421340, Rounding loss=7.648945)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=10.814922 (Recons loss=7.256758, Rounding loss=3.558164)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=8.160083 (Recons loss=7.739795, Rounding loss=0.420288)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=6.426042 (Recons loss=6.426042, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=6.168760 (Recons loss=6.168760, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.6/Add_output_0)->(/features/features.7/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.037159 to 0.036396 (diff=-0.000763)

36%|█████████████████████████████▍                          | 19/53 [08:55<09:40, 17.09s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.7/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.7/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=5.563629 (Recons loss=5.563629, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=4.325046 (Recons loss=4.325046, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=14.093557 (Recons loss=4.111204, Rounding loss=9.982354)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=8.819047 (Recons loss=4.741786, Rounding loss=4.077261)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=5.615909 (Recons loss=2.967691, Rounding loss=2.648218)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=6.312332 (Recons loss=4.674331, Rounding loss=1.638001)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=4.077104 (Recons loss=3.054713, Rounding loss=1.022391)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=3.591481 (Recons loss=3.249451, Rounding loss=0.342030)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=3.635466 (Recons loss=3.615466, Rounding loss=0.020000)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=6.059803 (Recons loss=6.059803, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=4.677032 (Recons loss=4.677032, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.7/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.7/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.022422 to 0.021995 (diff=-0.000427)

38%|██████████████████████████████▉                          | 20/53 [09:12<09:24, 17.11s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.7/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.7/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=40.762615 (Recons loss=40.762615, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=34.658955 (Recons loss=34.658955, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=129.311722 (Recons loss=40.171711, Rounding loss=89.140015)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=88.277908 (Recons loss=43.096241, Rounding loss=45.181664)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=70.787811 (Recons loss=38.022400, Rounding loss=32.765408)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=58.547832 (Recons loss=34.932205, Rounding loss=23.615627)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=53.812241 (Recons loss=38.472572, Rounding loss=15.339669)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=49.144188 (Recons loss=40.324528, Rounding loss=8.819659)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=91.869911 (Recons loss=88.443283, Rounding loss=3.426629)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=35.316814 (Recons loss=34.928646, Rounding loss=0.388167)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=45.552105 (Recons loss=45.546627, Rounding loss=0.005477)

 

[QUARK-INFO]: Module (/features/features.7/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.7/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.684990 to 0.644704 (diff=-0.040286)

40%|████████████████████████████████▍                         | 21/53 [09:26<08:35, 16.12s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.7/conv/conv.2/Conv_output_0)->(/features/features.8/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=7.252270 (Recons loss=7.252270, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=6.227381 (Recons loss=6.227381, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=169.392136 (Recons loss=5.790362, Rounding loss=163.601776)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=65.021774 (Recons loss=7.681398, Rounding loss=57.340378)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=41.762882 (Recons loss=7.789132, Rounding loss=33.973751)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=25.598701 (Recons loss=6.421778, Rounding loss=19.176924)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=14.836545 (Recons loss=6.024035, Rounding loss=8.812510)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=16.705153 (Recons loss=14.541611, Rounding loss=2.163541)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=6.404751 (Recons loss=6.305005, Rounding loss=0.099746)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=6.607412 (Recons loss=6.607412, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=5.986356 (Recons loss=5.986356, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.7/conv/conv.2/Conv_output_0)->(/features/features.8/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.018781 to 0.018368 (diff=-0.000413)

42%|██████████████████████████████████                        | 22/53 [09:41<08:02, 15.57s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.8/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.8/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=8.335863 (Recons loss=8.335863, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=5.360995 (Recons loss=5.360995, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=31.978241 (Recons loss=4.795360, Rounding loss=27.182880)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=18.497797 (Recons loss=3.743665, Rounding loss=14.754131)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=16.778267 (Recons loss=4.540265, Rounding loss=12.238003)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=23.293772 (Recons loss=12.920577, Rounding loss=10.373195)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=11.372615 (Recons loss=3.560619, Rounding loss=7.811996)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=9.830826 (Recons loss=4.852358, Rounding loss=4.978468)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=9.919705 (Recons loss=8.739470, Rounding loss=1.180235)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=3.526511 (Recons loss=3.506514, Rounding loss=0.019996)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=5.754101 (Recons loss=5.754101, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.8/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.8/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.014078 to 0.012115 (diff=-0.001963)

43%|███████████████████████████████████▌                       | 23/53 [09:56<07:44, 15.47s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.8/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.8/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=30.899673 (Recons loss=30.899673, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=10.105595 (Recons loss=10.105595, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=201.681213 (Recons loss=11.605256, Rounding loss=190.075958)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=109.858932 (Recons loss=11.245082, Rounding loss=98.613853)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=86.046516 (Recons loss=10.764030, Rounding loss=75.282486)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=70.635086 (Recons loss=15.381430, Rounding loss=55.253654)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=48.248108 (Recons loss=11.436881, Rounding loss=36.811226)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=32.350861 (Recons loss=12.749845, Rounding loss=19.601015)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=18.135813 (Recons loss=12.727210, Rounding loss=5.408602)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=13.851171 (Recons loss=13.591572, Rounding loss=0.259600)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=9.549848 (Recons loss=9.549433, Rounding loss=0.000415)

 

[QUARK-INFO]: Module (/features/features.8/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.8/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.248358 to 0.217527 (diff=-0.030831)

45%|█████████████████████████████████████▏                      | 24/53 [10:11<07:30, 15.52s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.8/Add_output_0)->(/features/features.9/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=28.477249 (Recons loss=28.477249, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=5.568766 (Recons loss=5.568766, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=177.474701 (Recons loss=6.158161, Rounding loss=171.316544)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=73.339249 (Recons loss=6.811993, Rounding loss=66.527252)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=46.317413 (Recons loss=6.279993, Rounding loss=40.037422)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=29.420017 (Recons loss=6.804386, Rounding loss=22.615631)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=16.796801 (Recons loss=6.078273, Rounding loss=10.718528)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=10.003111 (Recons loss=6.515441, Rounding loss=3.487669)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=7.989625 (Recons loss=7.757043, Rounding loss=0.232582)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=5.314819 (Recons loss=5.314819, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=6.956269 (Recons loss=6.956269, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.8/Add_output_0)->(/features/features.9/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.017520 to 0.017167 (diff=-0.000353)

47%|██████████████████████████████████████▋                      | 25/53 [10:26<07:10, 15.37s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.9/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.9/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=4.481857 (Recons loss=4.481857, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=4.986233 (Recons loss=4.986233, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=31.395355 (Recons loss=4.568803, Rounding loss=26.826551)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=18.805096 (Recons loss=4.054926, Rounding loss=14.750171)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=15.156006 (Recons loss=3.506808, Rounding loss=11.649199)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=12.501535 (Recons loss=3.342520, Rounding loss=9.159016)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=11.055155 (Recons loss=4.424176, Rounding loss=6.630979)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=7.555345 (Recons loss=3.680162, Rounding loss=3.875183)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=4.511865 (Recons loss=3.625447, Rounding loss=0.886417)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=3.832036 (Recons loss=3.825283, Rounding loss=0.006753)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=5.529317 (Recons loss=5.529317, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.9/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.9/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.012557 to 0.011550 (diff=-0.001007)

49%|████████████████████████████████████████▏                     | 26/53 [10:43<07:01, 15.61s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.9/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.9/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=14.387261 (Recons loss=14.387261, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=31.513256 (Recons loss=31.513256, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=201.065186 (Recons loss=14.799057, Rounding loss=186.266129)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=119.097733 (Recons loss=25.790400, Rounding loss=93.307335)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=91.051956 (Recons loss=20.769745, Rounding loss=70.282211)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=71.029655 (Recons loss=19.716681, Rounding loss=51.312973)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=53.452999 (Recons loss=18.597576, Rounding loss=34.855423)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=40.076180 (Recons loss=21.423721, Rounding loss=18.652460)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=25.514919 (Recons loss=20.467300, Rounding loss=5.047619)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=21.216385 (Recons loss=21.007183, Rounding loss=0.209202)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=17.409819 (Recons loss=17.407789, Rounding loss=0.002029)

 

[QUARK-INFO]: Module (/features/features.9/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.9/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.374775 to 0.339751 (diff=-0.035024)

51%|█████████████████████████████████████████▊                    | 27/53 [10:58<06:47, 15.67s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.9/Add_output_0)->(/features/features.10/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=5.085835 (Recons loss=5.085835, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=4.808451 (Recons loss=4.808451, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=178.652161 (Recons loss=7.977437, Rounding loss=170.674728)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=72.925606 (Recons loss=6.005297, Rounding loss=66.920311)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=46.771023 (Recons loss=6.454315, Rounding loss=40.316708)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=28.323753 (Recons loss=4.952341, Rounding loss=23.371412)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=15.672750 (Recons loss=5.335290, Rounding loss=10.337461)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=7.628469 (Recons loss=5.565031, Rounding loss=2.063438)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=5.143678 (Recons loss=5.073808, Rounding loss=0.069870)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=6.089964 (Recons loss=6.089964, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=5.756538 (Recons loss=5.756538, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.9/Add_output_0)->(/features/features.10/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.016455 to 0.016160 (diff=-0.000295)

53%|███████████████████████████████████████████▎                   | 28/53 [11:15<06:35, 15.83s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.10/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.10/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=3.216157 (Recons loss=3.216157, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=3.250889 (Recons loss=3.250889, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=26.056358 (Recons loss=2.668370, Rounding loss=23.387987)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=14.417672 (Recons loss=2.675078, Rounding loss=11.742594)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=13.494053 (Recons loss=4.266808, Rounding loss=9.227245)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=10.014308 (Recons loss=2.827779, Rounding loss=7.186529)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=8.239182 (Recons loss=2.842725, Rounding loss=5.396457)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=6.338953 (Recons loss=3.078589, Rounding loss=3.260364)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=4.266421 (Recons loss=3.472083, Rounding loss=0.794339)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=3.154462 (Recons loss=3.154462, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=2.932004 (Recons loss=2.932004, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.10/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.10/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.010357 to 0.009209 (diff=-0.001148)

55%|████████████████████████████████████████████▊                   | 29/53 [11:31<06:27, 16.15s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.10/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.10/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=23.457758 (Recons loss=23.457758, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=23.629028 (Recons loss=23.629028, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=224.338516 (Recons loss=33.080021, Rounding loss=191.258499)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=127.741287 (Recons loss=22.002426, Rounding loss=105.738861)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=105.735252 (Recons loss=21.989561, Rounding loss=83.745689)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=88.589539 (Recons loss=23.744053, Rounding loss=64.845490)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=64.550438 (Recons loss=18.385780, Rounding loss=46.164658)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=51.420818 (Recons loss=24.389900, Rounding loss=27.030918)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=54.862431 (Recons loss=44.106915, Rounding loss=10.755517)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=24.304712 (Recons loss=22.652277, Rounding loss=1.652435)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=21.561705 (Recons loss=21.452011, Rounding loss=0.109693)

 

[QUARK-INFO]: Module (/features/features.10/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.10/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.493988 to 0.404860 (diff=-0.089128)

57%|██████████████████████████████████████████████▍                  | 30/53 [11:48<06:16, 16.37s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.10/Add_output_0)->(/features/features.11/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=12.330524 (Recons loss=12.330524, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=10.914228 (Recons loss=10.914228, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=171.604065 (Recons loss=9.563097, Rounding loss=162.040970)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=74.026787 (Recons loss=13.707128, Rounding loss=60.319660)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=45.295372 (Recons loss=10.117413, Rounding loss=35.177959)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=28.823338 (Recons loss=9.709710, Rounding loss=19.113628)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=16.597206 (Recons loss=7.644488, Rounding loss=8.952719)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=19.874258 (Recons loss=17.385210, Rounding loss=2.489048)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=11.850098 (Recons loss=11.779195, Rounding loss=0.070903)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=12.796764 (Recons loss=12.796764, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=11.916418 (Recons loss=11.916418, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.10/Add_output_0)->(/features/features.11/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.032957 to 0.032424 (diff=-0.000533)

58%|███████████████████████████████████████████████▉                 | 31/53 [12:05<06:01, 16.43s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.11/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.11/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=7.851424 (Recons loss=7.851424, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=5.991529 (Recons loss=5.991529, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=34.865589 (Recons loss=7.438169, Rounding loss=27.427422)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=22.175714 (Recons loss=6.062019, Rounding loss=16.113695)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=19.166155 (Recons loss=5.473772, Rounding loss=13.692383)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=18.382494 (Recons loss=6.555820, Rounding loss=11.826674)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=14.167698 (Recons loss=4.625826, Rounding loss=9.541871)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=14.657624 (Recons loss=8.109215, Rounding loss=6.548409)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=9.676899 (Recons loss=7.470392, Rounding loss=2.206507)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=7.904022 (Recons loss=7.844513, Rounding loss=0.059509)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=7.138242 (Recons loss=7.138242, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.11/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.11/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.021707 to 0.020247 (diff=-0.001460)

60%|█████████████████████████████████████████████████▌                | 32/53 [12:24<06:01, 17.20s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.11/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.11/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=44.137432 (Recons loss=44.137432, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=48.976933 (Recons loss=48.976933, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=312.943939 (Recons loss=54.816990, Rounding loss=258.126953)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=148.769501 (Recons loss=47.253189, Rounding loss=101.516304)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=113.731812 (Recons loss=47.844143, Rounding loss=65.887672)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=89.544693 (Recons loss=47.187668, Rounding loss=42.357021)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=61.084045 (Recons loss=37.479877, Rounding loss=23.604170)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=62.642288 (Recons loss=53.370983, Rounding loss=9.271305)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=65.176773 (Recons loss=63.358074, Rounding loss=1.818698)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=43.010143 (Recons loss=42.962612, Rounding loss=0.047532)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=53.695969 (Recons loss=53.695969, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.11/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.11/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.547738 to 0.522619 (diff=-0.025119)

62%|███████████████████████████████████████████████████                | 33/53 [12:42<05:49, 17.46s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.11/conv/conv.2/Conv_output_0)->(/features/features.12/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=8.229621 (Recons loss=8.229621, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=11.677519 (Recons loss=11.677519, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=392.801239 (Recons loss=9.433465, Rounding loss=383.367767)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=132.098892 (Recons loss=9.085224, Rounding loss=123.013672)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=73.369652 (Recons loss=9.338771, Rounding loss=64.030884)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=38.498905 (Recons loss=9.409757, Rounding loss=29.089149)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=20.761986 (Recons loss=10.226213, Rounding loss=10.535773)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=13.224676 (Recons loss=11.130374, Rounding loss=2.094302)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=8.668374 (Recons loss=8.633334, Rounding loss=0.035039)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=9.414260 (Recons loss=9.414260, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=9.055752 (Recons loss=9.055752, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.11/conv/conv.2/Conv_output_0)->(/features/features.12/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.016900 to 0.016694 (diff=-0.000206)

64%|████████████████████████████████████████████████████▌               | 34/53 [13:02<05:44, 18.12s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.12/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.12/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=5.988861 (Recons loss=5.988861, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=5.059060 (Recons loss=5.059060, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=44.827602 (Recons loss=5.410549, Rounding loss=39.417053)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=27.493343 (Recons loss=7.607491, Rounding loss=19.885851)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=21.164883 (Recons loss=5.618332, Rounding loss=15.546550)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=18.175005 (Recons loss=6.177707, Rounding loss=11.997297)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=15.613691 (Recons loss=7.079296, Rounding loss=8.534395)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=9.964241 (Recons loss=5.616815, Rounding loss=4.347427)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=8.370617 (Recons loss=7.552619, Rounding loss=0.817998)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=7.328551 (Recons loss=7.328551, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=5.495011 (Recons loss=5.495011, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.12/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.12/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.011424 to 0.010946 (diff=-0.000478)

66%|██████████████████████████████████████████████████████▏              | 35/53 [13:25<05:54, 19.71s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.12/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.12/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=24.430250 (Recons loss=24.430250, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=25.215248 (Recons loss=25.215248, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=409.581909 (Recons loss=21.280228, Rounding loss=388.301697)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=162.222260 (Recons loss=24.590567, Rounding loss=137.631699)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=113.381920 (Recons loss=30.137327, Rounding loss=83.244591)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=69.150391 (Recons loss=22.541323, Rounding loss=46.609066)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=45.809357 (Recons loss=23.386400, Rounding loss=22.422958)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=30.711664 (Recons loss=23.577383, Rounding loss=7.134281)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=22.188852 (Recons loss=21.480894, Rounding loss=0.707958)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=22.264227 (Recons loss=22.256598, Rounding loss=0.007629)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=19.618704 (Recons loss=19.618704, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.12/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.12/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.257569 to 0.246172 (diff=-0.011398)

68%|███████████████████████████████████████████████████████▋             | 36/53 [13:46<05:40, 20.03s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.12/Add_output_0)->(/features/features.13/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=6.406298 (Recons loss=6.406298, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=8.164980 (Recons loss=8.164980, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=373.097504 (Recons loss=7.927369, Rounding loss=365.170135)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=107.573975 (Recons loss=7.486906, Rounding loss=100.087067)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=54.623314 (Recons loss=8.971005, Rounding loss=45.652309)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=23.401520 (Recons loss=7.261954, Rounding loss=16.139565)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=12.803886 (Recons loss=7.929878, Rounding loss=4.874008)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=9.408914 (Recons loss=8.712364, Rounding loss=0.696549)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=8.265328 (Recons loss=8.245328, Rounding loss=0.020000)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=10.808468 (Recons loss=10.808468, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=7.225443 (Recons loss=7.225443, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.12/Add_output_0)->(/features/features.13/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.014472 to 0.014346 (diff=-0.000126)

70%|█████████████████████████████████████████████████████████▏            | 37/53 [14:05<05:14, 19.65s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.13/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.13/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=4.677337 (Recons loss=4.677337, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=3.536747 (Recons loss=3.536747, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=43.623093 (Recons loss=4.113380, Rounding loss=39.509712)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=22.370663 (Recons loss=3.579127, Rounding loss=18.791536)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=17.577019 (Recons loss=3.149861, Rounding loss=14.427156)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=14.962398 (Recons loss=3.803635, Rounding loss=11.158763)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=11.667724 (Recons loss=3.893746, Rounding loss=7.773977)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=7.712867 (Recons loss=3.672191, Rounding loss=4.040676)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=4.055867 (Recons loss=3.373786, Rounding loss=0.682081)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=4.014573 (Recons loss=4.004589, Rounding loss=0.009984)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=4.450549 (Recons loss=4.450549, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.13/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.13/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.007636 to 0.007369 (diff=-0.000267)

72%|██████████████████████████████████████████████████████████▊            | 38/53 [14:25<04:56, 19.75s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.13/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.13/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=33.866726 (Recons loss=33.866726, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=33.039246 (Recons loss=33.039246, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=436.822937 (Recons loss=37.571388, Rounding loss=399.251556)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=186.082214 (Recons loss=24.952408, Rounding loss=161.129807)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=137.868164 (Recons loss=33.415764, Rounding loss=104.452400)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=98.332855 (Recons loss=32.788357, Rounding loss=65.544502)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=71.570740 (Recons loss=34.161411, Rounding loss=37.409332)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=45.898033 (Recons loss=29.261253, Rounding loss=16.636778)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=37.879562 (Recons loss=33.874519, Rounding loss=4.005042)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=37.981068 (Recons loss=37.943794, Rounding loss=0.037272)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=33.201508 (Recons loss=33.201508, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.13/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.13/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.380871 to 0.356165 (diff=-0.024706)

74%|████████████████████████████████████████████████████████████▎           | 39/53 [14:46<04:42, 20.15s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.13/Add_output_0)->(/features/features.14/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=6.649429 (Recons loss=6.649429, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=6.739950 (Recons loss=6.739950, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=392.983948 (Recons loss=8.130425, Rounding loss=384.853516)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=133.376846 (Recons loss=9.832118, Rounding loss=123.544724)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=70.110008 (Recons loss=8.553369, Rounding loss=61.556641)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=36.576515 (Recons loss=8.812953, Rounding loss=27.763563)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=17.420910 (Recons loss=7.920646, Rounding loss=9.500263)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=12.614738 (Recons loss=10.879972, Rounding loss=1.734766)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=6.897765 (Recons loss=6.870292, Rounding loss=0.027472)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=9.200605 (Recons loss=9.200605, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=7.116925 (Recons loss=7.116925, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.13/Add_output_0)->(/features/features.14/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.013897 to 0.013751 (diff=-0.000146)

75%|█████████████████████████████████████████████████████████████▉          | 40/53 [15:05<04:20, 20.02s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.14/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.14/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=5.458032 (Recons loss=5.458032, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=8.858489 (Recons loss=8.858489, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=42.999004 (Recons loss=5.527058, Rounding loss=37.471947)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=23.166496 (Recons loss=6.862724, Rounding loss=16.303772)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=17.083244 (Recons loss=5.893205, Rounding loss=11.190039)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=13.406359 (Recons loss=6.010113, Rounding loss=7.396245)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=10.202697 (Recons loss=6.365188, Rounding loss=3.837509)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=8.050254 (Recons loss=6.612840, Rounding loss=1.437413)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=6.676951 (Recons loss=6.500220, Rounding loss=0.176731)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=5.876461 (Recons loss=5.876461, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=7.026086 (Recons loss=7.026086, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.14/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.14/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.010752 to 0.010606 (diff=-0.000146)

77%|███████████████████████████████████████████████████████████████▍         | 41/53 [15:25<03:58, 19.90s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.14/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.14/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=45.663601 (Recons loss=45.663601, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=42.590626 (Recons loss=42.590626, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=748.436340 (Recons loss=49.749413, Rounding loss=698.686951)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=380.401276 (Recons loss=67.170662, Rounding loss=313.230621)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=264.485809 (Recons loss=52.083122, Rounding loss=212.402695)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=191.610184 (Recons loss=46.387356, Rounding loss=145.222824)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=124.094467 (Recons loss=36.673420, Rounding loss=87.421051)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=76.531891 (Recons loss=34.799889, Rounding loss=41.732002)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=48.257610 (Recons loss=38.781147, Rounding loss=9.476464)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=68.451454 (Recons loss=68.277313, Rounding loss=0.174144)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=49.685616 (Recons loss=49.685616, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.14/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.14/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.278205 to 0.262142 (diff=-0.016063)

79%|████████████████████████████████████████████████████████████████▉         | 42/53 [15:43<03:33, 19.44s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.14/conv/conv.2/Conv_output_0)->(/features/features.15/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=15.371315 (Recons loss=15.371315, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=13.948115 (Recons loss=13.948115, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=1096.219849 (Recons loss=15.649177, Rounding loss=1080.570679)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=372.480255 (Recons loss=14.280643, Rounding loss=358.199615)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=194.173996 (Recons loss=14.235291, Rounding loss=179.938705)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=89.929787 (Recons loss=10.679132, Rounding loss=79.250656)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=43.108398 (Recons loss=14.829831, Rounding loss=28.278568)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=19.234085 (Recons loss=14.122091, Rounding loss=5.111995)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=11.832145 (Recons loss=11.702246, Rounding loss=0.129899)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=11.426782 (Recons loss=11.426782, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=20.158693 (Recons loss=20.158693, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.14/conv/conv.2/Conv_output_0)->(/features/features.15/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.014803 to 0.014607 (diff=-0.000196)

81%|██████████████████████████████████████████████████████████████████▌        | 43/53 [16:02<03:11, 19.19s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.15/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.15/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=5.293988 (Recons loss=5.293988, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=5.129926 (Recons loss=5.129926, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=63.192684 (Recons loss=5.128208, Rounding loss=58.064476)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=28.692343 (Recons loss=3.735620, Rounding loss=24.956722)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=21.205240 (Recons loss=4.691385, Rounding loss=16.513855)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=15.234667 (Recons loss=4.684180, Rounding loss=10.550487)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=14.497168 (Recons loss=8.844646, Rounding loss=5.652522)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=6.105001 (Recons loss=3.955704, Rounding loss=2.149297)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=5.228429 (Recons loss=5.005127, Rounding loss=0.223302)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=5.142968 (Recons loss=5.142968, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=5.369523 (Recons loss=5.369523, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.15/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.15/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.006026 to 0.005656 (diff=-0.000370)

83%|████████████████████████████████████████████████████████████████████       | 44/53 [16:21<02:50, 19.00s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.15/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.15/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=22.317274 (Recons loss=22.317274, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=47.887646 (Recons loss=47.887646, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=1170.578491 (Recons loss=41.940540, Rounding loss=1128.637939)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=485.650391 (Recons loss=19.401455, Rounding loss=466.248932)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=318.260040 (Recons loss=25.476727, Rounding loss=292.783325)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=209.513763 (Recons loss=33.297691, Rounding loss=176.216064)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=121.737144 (Recons loss=26.934801, Rounding loss=94.802345)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=61.241730 (Recons loss=25.392031, Rounding loss=35.849697)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=33.449753 (Recons loss=27.636627, Rounding loss=5.813127)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=22.737844 (Recons loss=22.658087, Rounding loss=0.079758)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=27.599340 (Recons loss=27.599340, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.15/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.15/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.196815 to 0.176840 (diff=-0.019974)

85%|█████████████████████████████████████████████████████████████████████▌      | 45/53 [16:40<02:33, 19.25s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.15/Add_output_0)->(/features/features.16/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=10.984603 (Recons loss=10.984603, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=8.566879 (Recons loss=8.566879, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=1104.732300 (Recons loss=6.731795, Rounding loss=1098.000488)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=380.845673 (Recons loss=9.644191, Rounding loss=371.201477)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=207.725861 (Recons loss=9.070757, Rounding loss=198.655106)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=101.139580 (Recons loss=10.459200, Rounding loss=90.680382)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=44.082481 (Recons loss=13.572411, Rounding loss=30.510071)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=17.163403 (Recons loss=11.398771, Rounding loss=5.764631)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=9.698628 (Recons loss=9.581700, Rounding loss=0.116928)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=7.208968 (Recons loss=7.208968, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=5.988977 (Recons loss=5.988977, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.15/Add_output_0)->(/features/features.16/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.011168 to 0.010971 (diff=-0.000197)

87%|███████████████████████████████████████████████████████████████████████▏     | 46/53 [17:00<02:15, 19.34s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.16/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.16/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=2.284902 (Recons loss=2.284902, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=2.954323 (Recons loss=2.954323, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=70.439041 (Recons loss=1.545416, Rounding loss=68.893623)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=39.140530 (Recons loss=2.209747, Rounding loss=36.930782)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=31.576603 (Recons loss=2.042031, Rounding loss=29.534573)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=24.475153 (Recons loss=1.099977, Rounding loss=23.375175)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=19.740185 (Recons loss=2.333352, Rounding loss=17.406834)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=13.238569 (Recons loss=2.623329, Rounding loss=10.615240)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=6.178390 (Recons loss=2.655150, Rounding loss=3.523239)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=3.465255 (Recons loss=3.344302, Rounding loss=0.120953)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=2.769732 (Recons loss=2.769732, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.16/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.16/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.004792 to 0.003231 (diff=-0.001561)

89%|████████████████████████████████████████████████████████████████████████▋     | 47/53 [17:19<01:55, 19.27s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.16/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.16/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=69.449028 (Recons loss=69.449028, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=65.110077 (Recons loss=65.110077, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=1133.871704 (Recons loss=48.312618, Rounding loss=1085.559082)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=542.666077 (Recons loss=76.053032, Rounding loss=466.613037)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=356.594208 (Recons loss=50.834126, Rounding loss=305.760071)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=253.041519 (Recons loss=55.587048, Rounding loss=197.454468)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=185.935379 (Recons loss=65.374443, Rounding loss=120.560936)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=120.120155 (Recons loss=56.982391, Rounding loss=63.137764)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=79.349716 (Recons loss=56.032864, Rounding loss=23.316853)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=63.256657 (Recons loss=60.570637, Rounding loss=2.686020)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=73.945000 (Recons loss=73.928543, Rounding loss=0.016453)

 

[QUARK-INFO]: Module (/features/features.16/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.16/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.479321 to 0.383643 (diff=-0.095678)

91%|██████████████████████████████████████████████████████████████████████████▎    | 48/53 [17:40<01:38, 19.72s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.16/Add_output_0)->(/features/features.17/conv/conv.0/conv.0.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=15.087328 (Recons loss=15.087328, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=8.876220 (Recons loss=8.876220, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=1083.459229 (Recons loss=17.198925, Rounding loss=1066.260254)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=362.595398 (Recons loss=16.535284, Rounding loss=346.060120)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=188.480865 (Recons loss=13.138082, Rounding loss=175.342789)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=86.488419 (Recons loss=7.532298, Rounding loss=78.956123)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=41.109856 (Recons loss=12.806318, Rounding loss=28.303539)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=14.664244 (Recons loss=9.006354, Rounding loss=5.657890)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=12.389795 (Recons loss=12.210479, Rounding loss=0.179317)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=13.898338 (Recons loss=13.898338, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=7.933327 (Recons loss=7.933327, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.16/Add_output_0)->(/features/features.17/conv/conv.0/conv.0.2/Clip_output_0) recons metrics was optimized from 0.013262 to 0.013013 (diff=-0.000248)

92%|███████████████████████████████████████████████████████████████████████████▊   | 49/53 [18:00<01:19, 19.84s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.17/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.17/conv/conv.1/conv.1.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=7.332017 (Recons loss=7.332017, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=17.484379 (Recons loss=17.484379, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=69.200401 (Recons loss=16.304676, Rounding loss=52.895725)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=34.405121 (Recons loss=14.049915, Rounding loss=20.355206)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=35.113792 (Recons loss=23.183022, Rounding loss=11.930771)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=18.606897 (Recons loss=11.681562, Rounding loss=6.925335)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=15.482603 (Recons loss=12.317443, Rounding loss=3.165161)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=11.356132 (Recons loss=10.366570, Rounding loss=0.989561)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=11.147655 (Recons loss=11.060819, Rounding loss=0.086837)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=24.569008 (Recons loss=24.569008, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=13.805307 (Recons loss=13.805307, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.17/conv/conv.0/conv.0.2/Clip_output_0)->(/features/features.17/conv/conv.1/conv.1.2/Clip_output_0) recons metrics was optimized from 0.014111 to 0.013772 (diff=-0.000339)

94%|█████████████████████████████████████████████████████████████████████████████▎  | 50/53 [18:20<01:00, 20.05s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.17/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.17/conv/conv.2/Conv_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=80.080528 (Recons loss=80.080528, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=52.900028 (Recons loss=52.900028, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=2402.434570 (Recons loss=73.343369, Rounding loss=2329.091309)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=1142.681519 (Recons loss=90.920975, Rounding loss=1051.760498)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=789.040588 (Recons loss=88.106422, Rounding loss=700.934143)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=525.111572 (Recons loss=62.229733, Rounding loss=462.881866)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=350.710846 (Recons loss=69.683258, Rounding loss=281.027588)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=201.297806 (Recons loss=63.863586, Rounding loss=137.434219)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=101.996971 (Recons loss=65.330162, Rounding loss=36.666809)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=86.195023 (Recons loss=85.463058, Rounding loss=0.731964)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=65.587440 (Recons loss=65.587440, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.17/conv/conv.1/conv.1.2/Clip_output_0)->(/features/features.17/conv/conv.2/Conv_output_0) recons metrics was optimized from 0.258375 to 0.231064 (diff=-0.027310)

96%|██████████████████████████████████████████████████████████████████████████████▉  | 51/53 [18:43<00:41, 20.85s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/features/features.17/conv/conv.2/Conv_output_0)->(/features/features.18/features.18.2/Clip_output_0) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=52.693298 (Recons loss=52.693298, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=34.464920 (Recons loss=34.464920, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=2788.770752 (Recons loss=28.522568, Rounding loss=2760.248291)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=945.818359 (Recons loss=51.412113, Rounding loss=894.406250)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=471.536194 (Recons loss=25.050194, Rounding loss=446.485992)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=234.265869 (Recons loss=35.146404, Rounding loss=199.119461)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=86.321259 (Recons loss=14.089291, Rounding loss=72.231964)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=91.579437 (Recons loss=73.809235, Rounding loss=17.770206)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=64.519707 (Recons loss=63.241783, Rounding loss=1.277923)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=51.144272 (Recons loss=51.144272, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=37.210476 (Recons loss=37.210476, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/features/features.17/conv/conv.2/Conv_output_0)->(/features/features.18/features.18.2/Clip_output_0) recons metrics was optimized from 0.032653 to 0.031675 (diff=-0.000978)

98%|████████████████████████████████████████████████████████████████████████████████▍ | 52/53 [19:09<00:22, 22.29s/it]

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-WARNING]: Due to mismatch of dependent libraries, the custom op library from the pre-built wheel package failed to register with ORT. Please try to re-build the vai_q_onnx in current environment.

 

[QUARK-WARNING]: Not found extra options for FastFinetune, will use default parameters

 

[QUARK-INFO]: Module (/Flatten_output_0)->(output_QuantizeLinear_Input) will be optimized by adaround on cpu

 

[QUARK-INFO]: adaround iterations=0, lr=0.100000, loss=4.980998 (Recons loss=4.980998, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=100, lr=0.100000, loss=16.966026 (Recons loss=16.966026, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=200, lr=0.100000, loss=9399.071289 (Recons loss=8.199231, Rounding loss=9390.872070)

 

[QUARK-INFO]: adaround iterations=300, lr=0.100000, loss=3053.486816 (Recons loss=16.475918, Rounding loss=3037.010986)

 

[QUARK-INFO]: adaround iterations=400, lr=0.100000, loss=1538.883057 (Recons loss=10.559362, Rounding loss=1528.323730)

 

[QUARK-INFO]: adaround iterations=500, lr=0.100000, loss=668.619263 (Recons loss=19.445835, Rounding loss=649.173401)

 

[QUARK-INFO]: adaround iterations=600, lr=0.100000, loss=236.363205 (Recons loss=11.264566, Rounding loss=225.098633)

 

[QUARK-INFO]: adaround iterations=700, lr=0.100000, loss=59.630920 (Recons loss=11.212200, Rounding loss=48.418720)

 

[QUARK-INFO]: adaround iterations=800, lr=0.100000, loss=5.526903 (Recons loss=3.690737, Rounding loss=1.836166)

 

[QUARK-INFO]: adaround iterations=900, lr=0.100000, loss=11.848803 (Recons loss=11.848803, Rounding loss=0.000000)

 

[QUARK-INFO]: adaround iterations=999, lr=0.100000, loss=16.165726 (Recons loss=16.165726, Rounding loss=0.000000)

 

[QUARK-INFO]: Module (/Flatten_output_0)->(output_QuantizeLinear_Input) recons metrics was optimized from 0.012753 to 0.011747 (diff=-0.001006)

100%|██████████████████████████████████████████████████████████████████████████████████| 53/53 [19:40<00:00, 22.27s/it]

 

[QUARK-INFO]: ONNX inference costs 589.8s and Torch training costs 434.4s

 

[QUARK-INFO]: Finished running fast finetune for 53 modules.

The operation types and their corresponding quantities of the input float model is shown in the table below.

┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓

┃ Op Type       ┃ Float Model          ┃

┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩

│ Conv         │ 52              │

│ Constant       │ 70              │

│ Clip         │ 35              │

│ Add         │ 10              │

│ GlobalAveragePool  │ 1               │

│ Flatten       │ 1               │

│ Gemm         │ 1               │

├──────────────────────┼───────────────────────────────┤

│ Quantized model path │ models/mobilenetv2_quant.onnx │

└──────────────────────┴───────────────────────────────┘

The quantized information for all operation types is shown in the table below.

The discrepancy between the operation types in the quantized model and the float model is due to the application of graph optimization.

┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓

┃ Op Type      ┃ Activation ┃ Weights ┃ Bias   ┃

┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩

│ Conv       │ UINT8(52) │ INT8(52) │ INT8(52) │

│ Add        │ UINT8(10) │     │     │

│ GlobalAveragePool │ UINT8(1)  │     │     │

│ Flatten      │ UINT8(1)  │     │     │

│ Gemm       │ UINT8(1)  │ INT8(1) │ INT8(1) │

└───────────────────┴────────────┴──────────┴──────────┘

Model Size:

Float32 model size: 13.34 MB

Int8 quantized model size: 3.47 MB

Model Accuracy:

Evaluating: 100%|█████████████████████████████████████████████████████████████████| 1000/1000 [00:08<00:00, 114.18it/s]

Float32 model accuracy: Top1 0.713, Top5 0.906

Evaluating: 100%|██████████████████████████████████████████████████████████████████| 1000/1000 [00:22<00:00, 44.68it/s]

Int8 quantized model accuracy: Top1 0.716, Top5 0.900

path\to\quark_quantization

Evaluating: 100%|██████████████████████████████████████████████████████████████████| 1000/1000 [00:22<00:00, 44.41it/s]

Int8 quantized model accuracy (NPU): Top1 0.716, Top5 0.900

(venv) PS path\to\quark_quantization>

　続いて高度な量子化におけるMobileNetV2、INT8で--fast_finetuneオプション付きの量子化advanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnx --fast_finetuneを実行。

(venv) PS path\to\quark_quantization> python advanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnx --cross_layer_equalization

 

[QUARK-INFO]: Checking custom ops library ...

 

[QUARK-WARNING]: The custom ops library path\to\quark_quantization\venv\Lib\site-packages\quark\onnx\operators\custom_ops\lib\custom_ops.dll does NOT exist.

 

[QUARK-INFO]: Start compiling CPU version of custom ops library.

path\to\quark_quantization\venv\Lib\site-packages\torch\utils\cpp_extension.py:414: UserWarning: Error checking compiler version for cl: [WinError 2] 指定されたファイルが見つかりません。

 warnings.warn(f'Error checking compiler version for {compiler}: {error}')

INFO: Could not find files for the given pattern(s).

 

[QUARK-ERROR]: CPU version of custom ops library compilation failed:Command '['where', 'cl']' returned non-zero exit status 1.

 

[QUARK-WARNING]: Custom ops library compilation failed: CPU version of custom ops library compilation failed:Command '['where', 'cl']' returned non-zero exit status 1..

 

[QUARK-INFO]: Checked custom ops library.

The configuration of the quantization is Config(global_quant_config=QuantizationConfig(calibrate_method=<PowerOfTwoMethod.MinMSE: 1>, quant_format=<QuantFormat.QDQ: 1>, activation_type=<QuantType.QUInt8: 1>, weight_type=<QuantType.QInt8: 0>, input_nodes=[], output_nodes=[], op_types_to_quantize=[], nodes_to_quantize=[], extra_op_types_to_quantize=[], nodes_to_exclude=[], subgraphs_to_exclude=[], specific_tensor_precision=False, execution_providers=['CPUExecutionProvider'], per_channel=False, reduce_range=False, optimize_model=True, use_dynamic_quant=False, use_external_data_format=False, convert_fp16_to_fp32=False, convert_nchw_to_nhwc=False, include_sq=False, include_rotation=False, include_cle=True, include_auto_mp=False, include_fast_ft=False, enable_npu_cnn=True, enable_npu_transformer=False, debug_mode=False, print_summary=True, ignore_warnings=True, log_severity_level=1, extra_options={'ActivationSymmetric': True}))

[QUARK_INFO]: Time information:

2025-04-05 10:50:56.883296

[QUARK_INFO]: OS and CPU information:

                    system --- Windows

                     node --- gem12promax

                    release --- 11

                    version --- 10.0.26100

                    machine --- AMD64

                   processor --- AMD64 Family 25 Model 117 Stepping 2, AuthenticAMD

[QUARK_INFO]: Tools version information:

                    python --- 3.12.0

                     onnx --- 1.17.0

                  onnxruntime --- 1.20.1

                  quark.onnx --- 0.8+103c340fe2

[QUARK_INFO]: Quantized Configuration information:

                  model_input --- models/mobilenetv2-12.onnx

                 model_output --- models/mobilenetv2_quant.onnx

            calibration_data_reader --- <utils.ImageDataReader object at 0x000002433FBDB500>

             calibration_data_path --- None

                 quant_format --- QDQ

                  input_nodes --- []

                 output_nodes --- []

             op_types_to_quantize --- []

          extra_op_types_to_quantize --- []

                  per_channel --- False

                 reduce_range --- False

                activation_type --- QUInt8

                  weight_type --- QInt8

               nodes_to_quantize --- []

               nodes_to_exclude --- []

             subgraphs_to_exclude --- []

                optimize_model --- True

           use_external_data_format --- False

               calibrate_method --- PowerOfTwoMethod.MinMSE

              execution_providers --- ['CPUExecutionProvider']

                enable_npu_cnn --- True

            enable_npu_transformer --- False

           specific_tensor_precision --- False

                  debug_mode --- False

             convert_fp16_to_fp32 --- False

             convert_nchw_to_nhwc --- False

                  include_cle --- True

                  include_sq --- False

               include_rotation --- False

                include_fast_ft --- False

                 extra_options --- {'ActivationSymmetric': True}

 

[QUARK-INFO]: The input ONNX model models/mobilenetv2-12.onnx can create InferenceSession successfully

 

[QUARK-INFO]: Obtained calibration data with 1000 iters

 

[QUARK-INFO]: Removed initializers from input

 

[QUARK-INFO]: Simplified model sucessfully

 

[QUARK-INFO]: Duplicate the shared initializers in the model for separate quantization use across different nodes!

 

[QUARK-INFO]: Loading model...

 

[QUARK-INFO]: The input ONNX model C:/Users/reg/AppData/Local/Temp/vai.cpinit.bhyolrsx/model_cpinit.onnx can run inference successfully

 

[QUARK-INFO]: Start CrossLayerEqualization...

 

[QUARK-INFO]: CrossLayerEqualization pattern num: 2

 

[QUARK-INFO]: Total CrossLayerEqualization steps: 1

 

[QUARK-INFO]: CrossLayerEqualization Done.

 

[QUARK-INFO]: optimize the model for better hardware compatibility.

 

[QUARK-WARNING]: The opset version is 17 < 20. Skipping fusing Gelu.

 

[QUARK-INFO]: Start calibration...

 

[QUARK-INFO]: Start collecting data, runtime depends on your model size and the number of calibration dataset.

 

[QUARK-INFO]: Finding optimal threshold for each tensor using PowerOfTwoMethod.MinMSE algorithm ...

 

[QUARK-INFO]: Use all calibration data to calculate min mse

Computing range: 100%|█████████████████████████████████████████████████| 101/101 [14:59<00:00, 8.91s/tensor]

 

[QUARK-INFO]: Finished the calibration of PowerOfTwoMethod.MinMSE which costs 1003.7s

 

[QUARK-INFO]: Remove QuantizeLinear & DequantizeLinear on certain operations(such as conv-relu).

 

[QUARK-INFO]: Rescale GlobalAveragePool /GlobalAveragePool with factor 1.0048828125 to simulate DPU behavior.

 

[QUARK-INFO]: Adjust the quantize info to meet the compiler constraints

 

[QUARK-INFO]: Input pos of pooling layer /GlobalAveragePool is 4. Output pos of pooling layer /GlobalAveragePool is 5.Modify opos from 5 to 4.

 

[QUARK-INFO]: Adjust the quantize info to meet the compiler constraints

The operation types and their corresponding quantities of the input float model is shown in the table below.

┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓

┃ Op Type       ┃ Float Model          ┃

┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩

│ Conv         │ 52              │

│ Constant       │ 70              │

│ Clip         │ 35              │

│ Add         │ 10              │

│ GlobalAveragePool  │ 1               │

│ Flatten       │ 1               │

│ Gemm         │ 1               │

├──────────────────────┼───────────────────────────────┤

│ Quantized model path │ models/mobilenetv2_quant.onnx │

└──────────────────────┴───────────────────────────────┘

The quantized information for all operation types is shown in the table below.

The discrepancy between the operation types in the quantized model and the float model is due to the application of graph optimization.

┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓

┃ Op Type      ┃ Activation ┃ Weights ┃ Bias   ┃

┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩

│ Conv       │ UINT8(52) │ INT8(52) │ INT8(52) │

│ Add        │ UINT8(10) │     │     │

│ GlobalAveragePool │ UINT8(1)  │     │     │

│ Flatten      │ UINT8(1)  │     │     │

│ Gemm       │ UINT8(1)  │ INT8(1) │ INT8(1) │

└───────────────────┴────────────┴──────────┴──────────┘

Model Size:

Float32 model size: 13.34 MB

Int8 quantized model size: 3.45 MB

Model Accuracy:

Evaluating: 100%|███████████████████████████████████████████████████████| 1000/1000 [00:07<00:00, 131.26it/s]

Float32 model accuracy: Top1 0.713, Top5 0.906

Evaluating: 100%|████████████████████████████████████████████████████████| 1000/1000 [00:21<00:00, 46.89it/s]

Int8 quantized model accuracy: Top1 0.632, Top5 0.858

path\to\quark_quantization

Evaluating: 100%|████████████████████████████████████████████████████████| 1000/1000 [00:21<00:00, 47.56it/s]

Int8 quantized model accuracy (NPU): Top1 0.632, Top5 0.858

(venv) PS path\to\quark_quantization>

 

　最後に高度な量子化におけるMobileNetV2、INT8での --cross_layer_equalizationオプション付きのadvanced_quark_quantize.py --model_input models/mobilenetv2-12.onnx --model_output models/mobilenetv2_quant.onnx --cross_layer_equalizationを実行。

【オプションなしの量子化結果/MobileNetV2、INT8】

 

Model Size:

Float32 model size: 13.34 MB

Int8 quantized model size: 3.45 MB

Model Accuracy:

Evaluating: 100%|███████████████████████████████████████████████████████| 1000/1000 [00:07<00:00, 127.30it/s]

Float32 model accuracy: Top1 0.713, Top5 0.906

Evaluating: 100%|████████████████████████████████████████████████████████| 1000/1000 [00:22<00:00, 45.40it/s]

Int8 quantized model accuracy: Top1 0.642, Top5 0.869

path\to\quark_quantization

Evaluating: 100%|████████████████████████████████████████████████████████| 1000/1000 [00:21<00:00, 46.24it/s]

Int8 quantized model accuracy (NPU): Top1 0.642, Top5 0.869

 

 

 

 

 

【Fast Fine Tuneオプションによる量子化結果/MobileNetV2、INT8】

 

Model Size:

Float32 model size: 13.34 MB

Int8 quantized model size: 3.47 MB

Model Accuracy:

Evaluating: 100%|█████████████████████████████████████████████████████████████████| 1000/1000 [00:08<00:00, 114.18it/s]

Float32 model accuracy: Top1 0.713, Top5 0.906

Evaluating: 100%|██████████████████████████████████████████████████████████████████| 1000/1000 [00:22<00:00, 44.68it/s]

Int8 quantized model accuracy: Top1 0.716, Top5 0.900

path\to\quark_quantization

Evaluating: 100%|██████████████████████████████████████████████████████████████████| 1000/1000 [00:22<00:00, 44.41it/s]

Int8 quantized model accuracy (NPU): Top1 0.716, Top5 0.900

 

 

 

 

 

【Cross Layer Equalizationオプションによる量子化結果/MobileNetV2、INT8】

 

Model Size:

Float32 model size: 13.34 MB

Int8 quantized model size: 3.45 MB

Model Accuracy:

Evaluating: 100%|███████████████████████████████████████████████████████| 1000/1000 [00:07<00:00, 131.26it/s]

Float32 model accuracy: Top1 0.713, Top5 0.906

Evaluating: 100%|████████████████████████████████████████████████████████| 1000/1000 [00:21<00:00, 46.89it/s]

Int8 quantized model accuracy: Top1 0.632, Top5 0.858

path\to\quark_quantization

Evaluating: 100%|████████████████████████████████████████████████████████| 1000/1000 [00:21<00:00, 47.56it/s]

Int8 quantized model accuracy (NPU): Top1 0.632, Top5 0.858

　結果となる部分だけ抜粋、と思ったら、ちょっと離れすぎて見づらいので...

【オプションなしの量子化結果/MobileNetV2、INT8】

 

Float32 model accuracy: Top1 0.713, Top5 0.906

Int8 quantized model accuracy: Top1 0.642, Top5 0.869

Int8 quantized model accuracy (NPU): Top1 0.642, Top5 0.869

 

 

 

【Fast Fine Tuneオプションによる量子化結果/MobileNetV2、INT8】

 

Float32 model accuracy: Top1 0.713, Top5 0.906

Int8 quantized model accuracy: Top1 0.716, Top5 0.900

Int8 quantized model accuracy (NPU): Top1 0.716, Top5 0.900

 

 

 

【Cross Layer Equalizationオプションによる量子化結果/MobileNetV2、INT8】

 

Float32 model accuracy: Top1 0.713, Top5 0.906

Int8 quantized model accuracy: Top1 0.632, Top5 0.858

Int8 quantized model accuracy (NPU): Top1 0.632, Top5 0.858

　こんな感じの方がよいかな。

　精度なので小数点値は、1を100としたパーセンテージということでしょう。

　Float32は全て同じ、CPUとNPUは、何れもオプションなしに比べてCross Layer Equalization、Fast Fine Tuneの順に精度が上ってますね。

　それぞれのオプションでCPUとNPUの精度は全く同一(あれ、もしかするとおかしいかも...)。

　Float32モデルの精度と比べても、INT8なCPU・NPU共にCross Layer Equalizationは少し精度が落ちる程度、Fast Fine Tuneは、精度がより高くなっていますね。

　ちなみに今回もタスクマネージャは開きつつ、実行したものの、ながら作業をしていたりで、よく見ていませんでした。

関連リンク

2004-2008-someday