Inner Product / Fully Connected Layer

The InnerProduct layer (also usually referred to as the fully connected layer) treats the input as a simple vector and produces an output in the form of a single vector (with the blob’s height and width set to 1).


  • Parameters (InnerProductParameter inner_product_param)
    • Required
      • num_output (c_o): the number of filters
    • Strongly recommended
      • weight_filler [default type: 'constant' value: 0]
    • Optional
      • bias_filler [default type: 'constant' value: 0]
      • bias_term [default true]: specifies whether to learn and apply a set of additive biases to the filter outputs
  • From ./src/caffe/proto/caffe.proto:
message InnerProductParameter {
  optional uint32 num_output = 1; // The number of outputs for the layer
  optional bool bias_term = 2 [default = true]; // whether to have bias terms
  optional FillerParameter weight_filler = 3; // The filler for the weight
  optional FillerParameter bias_filler = 4; // The filler for the bias

  // The first axis to be lumped into a single inner product computation;
  // all preceding axes are retained in the output.
  // May be negative to index from the end (e.g., -1 for the last axis).
  optional int32 axis = 5 [default = 1];
  // Specify whether to transpose the weight matrix or not.
  // If transpose == true, any operations will be performed on the transpose
  // of the weight matrix. The weight matrix itself is not going to be transposed
  // but rather the transfer flag of operations will be toggled accordingly.
  optional bool transpose = 6 [default = false];

Dialogue & Discussion