Stack Overflow
Stack Overflow

Questions tagged [quantization]

Use this tag for questions related to quantization of any kind, such as vector quantization.

Quantization, in mathematics and digital signal processing, is the process of mapping a large set of input values to a (countable) smaller set.

For more, please read the Wikipedia article for more.

368 questions
18
votes
3 answers

Quantize a Keras neural network model

Recently, I've started creating neural networks with Tensorflow + Keras and I would like to try the quantization feature available in Tensorflow. So far, experimenting with examples from TF tutorials worked just fine and I have this basic working…
sikr_
  • 312
  • 1
  • 5
  • 14
15
votes
2 answers

Generate color palette from image with ImageMagick

I need to generate a color palette of the top 5 dominant colors in an image. I'd like to replicate the results of Embedly's extract tool or Color Thief. The following command gives me slightly different results: convert testimage.jpg -format %c…
Ben Foster
  • 32,767
  • 35
  • 157
  • 274
13
votes
2 answers

Color banding only on Android 4.0+

On emulators running Android 4.0 or 4.0.3, I am seeing horrible colour banding which I can't seem to get rid of. On every other Android version I have tested, gradients look smooth. I have a SurfaceView which is configured as RGBX_8888, and the…
threeshinyapples
  • 181
  • 1
  • 5
10
votes
2 answers

Convert/Quantize Float Range to Integer Range

Say I have a float in the range of [0, 1] and I want to quantize and store it in an unsigned byte. Sounds like a no-brainer, but in fact it's quite complicated: The obvious solution looks like this: unsigned char QuantizeFloat(float a) { return…
Nils Pipenbrinck
  • 77,289
  • 24
  • 142
  • 216
10
votes
0 answers

Is it possible to use tf.contrib.quantize.create_training_graph with Keras model?

Is it possible to use tf.contrib.quantize.create_training_graph for model quantiation with already trained Keras model? As I understand I can import tf.Graph from Keras model, but can I finetune it after modification with…
mrgloom
  • 15,245
  • 23
  • 126
  • 226
10
votes
3 answers

How to calculate audio file size?

You have 30 seconds audio file sampled at a rate of 44.1 KHz and quantized using 8 bits ; calculate the bit rate and the size of mono and stereo versions of this file ؟؟
Osama Al-far
  • 259
  • 3
  • 6
  • 17
9
votes
2 answers

Understanding tf.contrib.lite.TFLiteConverter quantization parameters

I'm trying to use UINT8 quantization while converting tensorflow model to tflite model: If use post_training_quantize = True, model size is x4 lower then original fp32 model, so I assume that model weights are uint8, but when I load model and get…
mrgloom
  • 15,245
  • 23
  • 126
  • 226
8
votes
1 answer

tflite quantized inference very slow

I am trying to convert a trained model from checkpoint file to tflite. I am using tf.lite.LiteConverter. The float conversion went fine with reasonable inference speed. But the inference speed of the INT8 conversion is very slow. I tried to debug by…
wuhy08
  • 321
  • 2
  • 5
8
votes
4 answers

Edge TPU Compiler: ERROR: quantized_dimension must be in range [0, 1). Was 3

I'm trying to get a Mobilenetv2 model (retrained last layers to my data) to run on the Google edge TPU Coral. I've followed this tuturial https://www.tensorflow.org/lite/performance/post_training_quantization?hl=en to do the post-training…
Paulo Ribeiro
  • 145
  • 8
8
votes
0 answers

How can I quantize facenet's Inception-ResNet-v1 model in Tensorflow?

What I want to do I'm trying to create a quantized version of the Inception-ResNet-v1 model used in facenet - with not only quantized weights, but quantized nodes as well, according to Tensorflow's graph_transform guide. What I have tried Using a…
hannabe
  • 136
  • 4
7
votes
0 answers

Why my Tensorflow network becomes slower after applying the Quantization tools on GeForce GTX 1080?

I tried the quantization tools in Tensorflow with a toy model. It surely reduced the model to about 25% percent, however, increase the executing time by many times. GPU is fully used when both model runs. So I am wondering what is wrong? I guess…
samuel
  • 246
  • 2
  • 8
7
votes
1 answer

Generate the Dominant Colors for an RGB image with XMLHttpRequest

A Note For Readers: This is a long question, but it needs a background to understand the question asked. The color quantization technique is commonly used to get the dominant colors of an image. One of the well-known libraries that do color…
loretoparisi
  • 12,864
  • 9
  • 78
  • 108
7
votes
1 answer

Matlab : Unable to get unique rationals when implementing a formula for binary to real number conversion Part1

There is a nonlinear dynamic system x_n = f(x_n,eta) whose functional form is x[n+1] = 2*x[n] mod 1. This is a chaotic dynamical system called as the Sawtooth map or the Bernoulli Map. I am facing difficulty in implementing the two representations…
SKM
  • 919
  • 2
  • 15
  • 39
7
votes
3 answers

Effective gif/image color quantization?

So I'm trying to encode some animated gif files in my Java application. I've been using some classes/algorithms found online, but none seem to be working well enough. Right now I'm using this quantize class to reduce the colors of an image down to…
FTLRalph
  • 894
  • 8
  • 27
6
votes
1 answer

How can I matrix-multiply two PyTorch quantized Tensors?

I am new to tensor quantization, and tried doing something as simple as import torch x = torch.rand(10, 3) y = torch.rand(10, 3) x@y.T with PyTorch quantized tensors running on CPU. I thus tried scale, zero_point = 1e-4, 2 dtype = torch.qint32 qx…
Davide Fiocco
  • 3,631
  • 1
  • 23
  • 53
1
2 3
…
24 25