• Intro
• SE and CE
• Gradient
• Hiden
• Codes

What's the difference between cross-entropy and mean-squared-error loss functions?

These loss functions have different derivatives and different purposes. From a probabilistic point of view, the cross-entropy arises as the natural cost function to use if you have a sigmoid or softmax nonlinearity in the output layer of your network, and you want to maximize the likelihood of classifying the input data correctly.

If instead you assume the target is continuous and normally distributed, and you maximize the likelihood of the output of the net under these assumptions, you get the MSE (combined with a linear output layer).

For classification, cross-entropy tends to be more suitable than MSE -- the underlying assumptions just make more sense for this setting. That said, you can train a classifier with the MSE loss and it will probably work fine (although it does not play very nicely with the sigmoid/softmax nonlinearities, a linear output layer would be a better choice in that case). For regression problems, you would almost always use the MSE.

Another alternative for classification is to use a margin loss, which basically amounts to putting a (linear) SVM on top of your network.

• 神经网络
  • 简图
  • 描述
  • 梯度

RAWCODE

• 统计学
• 数学操作符
• 机器学习

RAWCODE

• 百科
  • 相关概念
  • 交叉熵
  • 与似然的关系
• 交叉熵损失函数
  • 定义符号
  • Logistic回归的交叉熵
  • Softmax回归的交叉熵
• 参考

• 总体设计
• 日志系统
• 消息系统
• 引擎设计
  • 模型
    • 设备模型
    • 规则模型
  • CLP语言
  • RulePayload设计
  • CLP设计
  • 引擎UML
• HttpHandler
• 调试系统
  • 系统监控(Python)
  • 规则配置(JS)

网上很多写卷积的文档都是用"很简单"表示卷积so easy, 但草民认真读了他们的文章后, 反而对 卷积更不理解, 即使到现在还是不能理解, 这篇文章只是从数据上分析卷积过程, 很多疑点还需继 续探究

1 参考

http://louistiao.me/posts/notebooks/embedding-matplotlib-animations-in-jupyter-notebooks/

http://louistiao.me/posts/notebooks/embedding-matplotlib-animations-in-jupyter-as-interactive-javascript-widgets/

非常形象的讲解张量

youtube

知乎部分翻译

一个简单的小程序, 来了解感知机如何工作的, 激活函数为sign