Learning deep learning (project 2, image classification)

1 min read

In this class project, I built a network to classify images in the CIFAR-10 dataset. This dataset is freely available.

The dataset contains 60K color images (32×32 pixel) in 10 classes, with 6K images per class.

Here are the classes in the dataset, as well as 10 random images from each:

airplane
automobile
bird
cat
deer
dog
frog
horse
ship
truck

You can imagine it’s not possible to write down all rules to classify them, so we have to write a program which can learn.

The neural network I created contains 2 hidden layers. The first one is a convolutional layer with max pooling. Then drop out 70% of the connections. The second layer is a fully connected layer with 384 neurons.

def conv_net(x, keep_prob):
    """
    Create a convolutional neural network model
    : x: Placeholder tensor that holds image data.
    : keep_prob: Placeholder tensor that hold dropout keep probability.
    : return: Tensor that represents logits
    """
    # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
    #    Play around with different number of outputs, kernel size and stride
    # Function Definition from Above:
    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    model = conv2d_maxpool(x, conv_num_outputs=18, conv_ksize=(4,4), conv_strides=(1,1), pool_ksize=(8,8), pool_strides=(1,1))
    model = tf.nn.dropout(model, keep_prob)

    # TODO: Apply a Flatten Layer
    # Function Definition from Above:
    #   flatten(x_tensor)
    model = flatten(model)

    # TODO: Apply 1, 2, or 3 Fully Connected Layers
    #    Play around with different number of outputs
    # Function Definition from Above:
    #   fully_conn(x_tensor, num_outputs)
    model = fully_conn(model,384)

    model = tf.nn.dropout(model, keep_prob)

    # TODO: Apply an Output Layer
    #    Set this to the number of classes
    # Function Definition from Above:
    #   output(x_tensor, num_outputs)
    model = output(model,10)

    # TODO: return output
    return model

Then I trained this network using Amazon AWS g2.2xlarge instance. This instance has GPU which is much faster for deep learning (than CPU). I did a simple experiment and find GPU is at least 3 times faster than CPU:

if all layers in gpu: 14 seconds to run 4 epochs,
if conv layer in cpu, other gpu, 36 seconds to run 4 epochs

This is apparently a very crude comparison but GPU is definitely much faster than CPU (at least the ones in AWS g2.2xlarge, cost: $0.65/hour)

Eventually I got ~70% accuracy on the test data, much better than random guess (10%). The time to train the model is ~30 minutes.

You can find my entire code at:
https://www.alivelearn.net/deeplearning/dlnd_image_classification_submission2.html

第十九期 fNIRS Journal Club 通知 2021/05/29,9:30am

美国普渡大学童云杰助理教授,将为大家讲解他们组最近被接受的一篇使用近红外相位信息研究脑血流变化的文章。热烈欢迎大家参与讨论。 时间: 北京时间2021年5月29日上午9:30地点: https://zoom.com房间号: 846 8391 7517密码: 805190 童云杰教授简介:普渡大学 生物医学工程助理教授、博士生导师。主攻方向是多模态脑成像, 包括核磁,fNIRS, EEG。关注脑功能及生理信号的提取与研究。发表论文九十余篇,引用上千次(H-index = 20)。 童教授要讲解的文章如下: Liang Z, Tian H, Yang HC, Arimitsu T, Takahashi...
Xu Cui
12 sec read

第十八期 fNIRS Journal Club 视频

北京时间2021年4月25日10点,北京师范大学的朱朝喆教授为大家讲解了他们最近几年在经颅脑图谱(Transcranial brain Atlas) 方面做的工作。视频如下: Youtube: https://youtu.be/EhYPuBPQ5uI Youku: 该视频在优酷上传后被优酷屏蔽,不清楚什么原因。申诉无效。
Xu Cui
3 sec read

第七届全国近红外光谱脑功能成像学术会议

会议日期:2021年5月22日-24日会议地点:天津师范大学 一、 会议简介       近红外光谱脑功能成像(fNIRS)具有设备购买与使用成本低、可在自然环境条件下使用、具有较高的时间分辨率和空间定位能力等特点,受到了脑科学研究的高度重视。“近红外光谱脑功能成像学术会议”是由北京师范大学认知神经科学与学习国家重点实验室朱朝喆教授发起并组织的全国性学术会议。已连续成功举办六届,共吸引全国近百家高校、科研院所及医院的六百余名学者参加。该会议已成为国内规模和影响力最大的fNIRS脑成像学术活动。       本届会议由北京师范大学与天津师范大学联合主办。会议将延用往届会议将学术报告与研究方法工作坊相结合的模式。学术报告模块(5月22日周六)将邀请心理学与认知神经科学领域、基础与临床医学领域以及工程技术领域知名学者汇报其fNIRS最新研究成果;工作坊模块(5月23-24日)由fNIRS领域一线研究者系统讲授fNIRS成像原理、fNIRS实验设计、fNIRS数据分析与统计、fNIRS论文写作以及fNIRS前沿技术等。除理论讲授外,还设置了fNIRS空间定位与数据分析操作(NIRS-KIT软件)环节,此外还安排充足的研讨答疑时间以便与会人员交流互动。       具体日程与详细内容等最新消息请关注后续通知,可通过天津师范大学心理部网站http://psych.tjnu.edu.cn/或北京师范大学国家重点实验室网站http://brain.bnu.edu.cn/,或者扫描下方二维码关注微信公众号-“fNIRS脑成像实验室”查阅更新信息,期盼在天津师范大学与您相聚! 二、会议组织机构 主办单位:教育部人文社会科学重点研究基地天津师范大学心理与行为研究院、天津师范大学心理学部、北京师范大学认知神经科学与学习国家重点实验室会议主席:白学军、朱朝喆组织委员会:赵春健、杨邵峰、侯鑫、曹正操 三、说明1.        学术报告模块注册费:人民币500元/人;工作坊模块注册费:人民币2500元/人。发票为电子发票,内容均为:“会议费”。两个模块各自独立收费,参会者可根据自己需要进行选择。2.        注册费包括各自模块的资料费、午餐费;其他费用自理。3.        会议报告人免除会议模块注册费,其他费用请自理。4.       ...
Xu Cui
18 sec read

2 Replies to “Learning deep learning (project 2, image classification)”

  1. Helpful post. Can you explain your motivation behind using standard deviation on 0.1 while initializing the weights. My network does not learn if i keep the standard deviation to 1. Only when i saw your post and fine tuned my standard deviation to 0.1, it started training. i would like to understand how did you choose the standard deviation of 0.1 🙂

  2. Can you explain how you arrived at the values below?

    model = fully_conn(model,384)
    #model = fully_conn(model,200)
    #model = fully_conn(model,20)

Leave a Reply to kushal Cancel reply

Your email address will not be published. Required fields are marked *

Loading