Hadoop: Distributed Cluster, YARN, MapReduce

This document is a tutorial on how to set up a distributed Hadoop cluster. It contains steps necessary for minimal working configuration for the cluster in the simplest scenario - launching an example program. Remember, Hadoop applies the parameters specified in configuration only when those parameters are used. Consequently, parameters that work for one task will not necessarily work for another, make sure to recheck all parameters for new tasks. In the case of incorrect parameters, the execution of a Hadoop program will be terminated and the error details will be written to a log file. Consult with the log file to identify the error cause. There is a troubleshooting section at the end of this document, where you can find solutions for most frequent problems.

Word2Vec

The performance of ML algorithms depends on the quality of the input data. That is, we want the input representation to be sufficient to represent some properties of the input itself. Problems with input representation are especially evident in the case of discrete features. The classical approach for dealing with discrete features is one-hot embeddings.

Working With Vagrant

This note will provide you some background on how to deploy virtual machines with Vagrant. More specifically, we will have a look at the basic outline and features of Vagrantfile . This tutorial is a distilled version of the official Vagrant Documentation, and most of the examples are borrowed from there.

Facial Recognition

The classical way to approach the problem of face recognition is to collect the data with different faces of people and build a classifier that will tell the difference between people. The problem with this approach is that such models are not very portable. Adding new people requires retraining the model and the behavior on people not present in the dataset is not well defined. Another approach to this problem is Unconstrained Classification where the number of target classes is not specified at the training time.

Loading Saved Graph

```python

assemble graph firts, names should match

with tf.Session() as sess: with slim.arg_scope(inception_resnet_v2_arg_scope()): saver = tf.train.Saver() saver.restore(sess, ckpt_path) sess.graph.as_default()

Pagination


© 2022. Vitaly Romanov

Powered by Hydejack v8.1.1