<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>HY's Farm</title>
    <description></description>
    <link>https://cultivo-hy.github.io/</link>
    <atom:link href="https://cultivo-hy.github.io/feed.xml" rel="self" type="application/rss+xml"/>
    <pubDate>Tue, 17 Sep 2019 05:07:40 +0000</pubDate>
    <lastBuildDate>Tue, 17 Sep 2019 05:07:40 +0000</lastBuildDate>
    <generator>Jekyll v3.8.5</generator>
    
      <item>
        <title>[Graph] 응용 그래프(Applied Graph Theory)</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#응용그래프-applied-graph-theory-중간고사-최종-보고서&quot; id=&quot;markdown-toc-응용그래프-applied-graph-theory-중간고사-최종-보고서&quot;&gt;응용그래프 (Applied Graph Theory) 중간고사 최종 보고서&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#코드&quot; id=&quot;markdown-toc-코드&quot;&gt;코드&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;응용그래프-applied-graph-theory-중간고사-최종-보고서&quot;&gt;응용그래프 (Applied Graph Theory) 중간고사 최종 보고서&lt;/h1&gt;

&lt;h5 id=&quot;코드&quot;&gt;코드&lt;/h5&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;import sys
import matplotlib.pyplot as plt
import numpy as np
import networkx as nx
import math
import concavehull as cv
from scipy.spatial import Delaunay
from scipy.spatial import ConvexHull
from itertools import product

numbed_of_point = 0         # The number of point
threshold = 0         # Threshold value ( vertex to cycle )
point_list = []          # Point [ [x1, y1], [x2, y2], ...]  list
cycle_vertex_list = []          # Cycle vertex [ v1, v2, .. ] list
cycle_length = 1000 * 1000    # The Length of Cycle
center= 0                     # center of the cycle
final_center = 0

city_f = open('city-100.txt','r')
cycle_f = open('citycycle-100.txt','w')

def getDistance(first_vertex, second_vertex):
    tmp1, tmp2 = point_list[first_vertex], point_list[second_vertex]
    x_diff , y_diff = tmp2[0] - tmp1[0] , tmp2[1] - tmp1[1]
    return round(math.sqrt(x_diff**2+y_diff**2),3)

def get_vertex_num(tpoint):
    for iter_i in range(len(point_list)):
        if point_list[iter_i] == tpoint:
            return iter_i

def get_cycle_length(cycle_list_parameter):
    cycle_lens = 0
    for cycle_list_iter in range(len(cycle_list_parameter) - 1):
        tmp_start, tmp_end = cycle_list_parameter[cycle_list_iter], cycle_list_parameter[cycle_list_iter + 1]
        cycle_lens += edgeLengthList[tmp_start - 1][tmp_end - 1]
    cycle_lens = round(cycle_lens, 3)
    return cycle_lens

def get_inserted_concave(target_value, concave_vertex_parameter, cycle_make_from_to_parameter):
    local_temp_cycle_candidate = []
    for count_index in range(len(concave_vertex_parameter)):
        temp_concave_vertex_list = concave_vertex_parameter.copy()
        temp_concave_vertex_list.insert(count_index,target_value)
        temporary_cycle, temporary_vertex = create_cycle_from_concave(temp_concave_vertex_list)
        temporary_cycle = remove_duplicated_vertex(temporary_cycle, temporary_vertex)
        temporary_cycle = optimize_concaved_cycle(temporary_cycle, temp_concave_vertex_list, cycle_make_from_to_parameter)
        temporary_cycle.append(temporary_cycle[0])
        temp_cycle_length = get_cycle_length(temporary_cycle)
        if temporary_cycle[0] == temporary_cycle[-1] and len(temporary_cycle[1:]) == len(set(temporary_cycle[1:])):
            local_temp_cycle_candidate.append([temp_concave_vertex_list,temporary_cycle, temp_cycle_length])
    if len(local_temp_cycle_candidate) != 0:
        tmp_result_list = sorted(local_temp_cycle_candidate, key=lambda x: x[2])[0]
        return_concave = []
        for cycle_it in tmp_result_list[1]:
            if cycle_it in tmp_result_list[0]:
                return_concave.append(cycle_it)
        tmp_result_list[0] = return_concave
        return tmp_result_list
    return 0

def create_cycle_from_concave(concave_parameter):
    new_cycle_list = []
    new_vertex_check_list = [0 for i in range(numbed_of_point)]
    for concave_iter in range(len(concave_parameter)-1):
        concave_start, concave_end = concave_parameter[concave_iter], concave_parameter[concave_iter+1]
        shortest_path_list = nx.shortest_path(Graph, concave_start, concave_end, weight='weight')
        for shortest_path_iter in range(len(shortest_path_list)-1):
            new_cycle_list.append(shortest_path_list[shortest_path_iter])
            new_vertex_check_list[shortest_path_list[shortest_path_iter]-1] += 1

    concave_start = concave_parameter[-1]   # center가 마지막 concave에 포함되는 경우 추가해줘야 함
    concave_end = concave_parameter[0]
    if concave_start != concave_end:
        shortest_path_list = nx.shortest_path(Graph, concave_start, concave_end, weight='weight')
        for shortest_path_iter in range(len(shortest_path_list) - 1):
            new_cycle_list.append(shortest_path_list[shortest_path_iter])
            new_vertex_check_list[shortest_path_list[shortest_path_iter] - 1] += 1

    return new_cycle_list, new_vertex_check_list

def remove_duplicated_vertex(cycle_list_parameter, vertex_check_list_parameter):
    for vertex_check_iter in range(len(vertex_check_list_parameter)):
        compare_path = []
        if vertex_check_list_parameter[vertex_check_iter] == 2:
            for c_ind in range(len(cycle_list_parameter)):
                if vertex_check_iter + 1 == cycle_list_parameter[c_ind]:
                    cycle_start = cycle_list_parameter[c_ind - 1]
                    cycle_end = cycle_list_parameter[c_ind + 1] if c_ind != len(cycle_list_parameter) - 1 else cycle_list_parameter[0]
                    compare_path.append([cycle_start, cycle_end, c_ind])

            if (compare_path[0][1] == compare_path[1][0] and compare_path[0][1] == vertex_check_iter + 1) or \
                    (compare_path[0][0] == compare_path[1][1] and compare_path[0][0] == vertex_check_iter + 1):
                del cycle_list_parameter[compare_path[0][2]]
            else:
                test = Graph.copy()
                remove_vertex = vertex_check_iter+1
                test.remove_node(remove_vertex)
                if remove_vertex != compare_path[0][0] and remove_vertex != compare_path[0][1] and \
                        remove_vertex != compare_path[1][0] and remove_vertex != compare_path[1][1]:
                    first_flag = nx.has_path(test, compare_path[0][0], compare_path[0][1])
                    second_flag = nx.has_path(test, compare_path[1][0], compare_path[1][1])
                    if first_flag == False and second_flag == False:
                        del cycle_list_parameter[compare_path[0][2]]
                    else:
                        removed_index = -1
                        if first_flag == True and second_flag == True:
                            first_shortest = nx.shortest_path_length(test, compare_path[0][0], compare_path[0][1],weight='weight')
                            second_shortest = nx.shortest_path_length(test, compare_path[1][0], compare_path[1][1], weight='weight')
                            removed_index = 0 if first_shortest &amp;lt; second_shortest else 1
                        elif first_flag:
                            removed_index = 0
                        else:
                            removed_index = 1
                        remove_path = nx.shortest_path(test, compare_path[removed_index][0], compare_path[removed_index][1], weight='weight')
                        remove_list = []
                        for remove_path_iter in range(1, len(remove_path) - 1):
                            remove_list.append(remove_path[remove_path_iter])
                        insert_index = compare_path[removed_index][2]
                        del cycle_list_parameter[insert_index]
                        for remove_list_iter in remove_list:
                            cycle_list_parameter.insert(insert_index, remove_list_iter)
                            insert_index += 1

    return cycle_list_parameter

def optimize_concaved_cycle(cycle_list_parameter, concave_vertex_parameter, cycle_make_from_to_parameter): # 최적화 가능한 concave_vertex 제거
    remove_list = []
    for concave_iter in concave_vertex_parameter:
        checklist = []
        for cycle_make_from_iter in cycle_make_from_to_parameter:
            if cycle_make_from_iter[1] == concave_iter:
                checklist.append(cycle_make_from_iter)
        if concave_iter in cycle_list_parameter:
            concave_index = cycle_list_parameter.index(concave_iter)
            for cycle_make_iter in cycle_make_from_to_parameter:
                vertex_from, vertex_to = cycle_make_iter[0], cycle_make_iter[1]
                if vertex_to == concave_iter:
                    temp_cycle_list = list(set(cycle_list_parameter) - set([concave_iter]))
                    for cycle_it in temp_cycle_list:
                        left = len(cycle_list_parameter)-1 if concave_index == 0 else concave_index - 1
                        right = 0 if concave_index == len(cycle_list_parameter)-1 else concave_index + 1
                        compare = [vertex_from, concave_iter]
                        if threshold &amp;gt; edgeLengthList[vertex_from - 1][cycle_it - 1] and Graph.has_edge(cycle_list_parameter[left], cycle_list_parameter[right]):
                            if compare in checklist:
                                checklist.remove([vertex_from, concave_iter])
                                if len(checklist) == 0:
                                    remove_list.append(concave_iter)
                                    break
    remove_list = list(set(remove_list))
    for remove_list_iter in remove_list:
        cycle_list_parameter.remove(remove_list_iter)
        concave_vertex_parameter.remove(remove_list_iter)
    return cycle_list_parameter

def is_all_vertex_in_threshold(cycle_parameter):
    threshold_result = True
    for i in range(1, len(point_list) + 1):
        if i in cycle_parameter: continue
        else:
            minimum_dist = 1000 * 1000
            for j in cycle_parameter:
                current_dist = nx.shortest_path_length(Graph,i,j,weight='weight')
                if current_dist &amp;lt; minimum_dist:
                    minimum_dist = current_dist
            if minimum_dist &amp;gt; threshold:
                threshold_result = False
                break
    return threshold_result

def get_optimal_cycle(cycle_parameter):
    tmp_cycle_parameter = cycle_parameter.copy()
    for cycle_rm_it in range(1, len(tmp_cycle_parameter)):
        is_removed = False
        neighbor_flag = False
        if cycle_rm_it == len(tmp_cycle_parameter)-1 or cycle_rm_it == len(tmp_cycle_parameter):
            neighbor_flag = Graph.has_edge(tmp_cycle_parameter[cycle_rm_it -1], tmp_cycle_parameter[1])
            if neighbor_flag:
                del tmp_cycle_parameter[0]  # 처음 끝 제거
                del tmp_cycle_parameter[-1]  
                tmp_cycle_parameter.append(tmp_cycle_parameter[0]) # 마지막 꺼 추가
                is_removed = True
            break
        else:
            neighbor_flag = Graph.has_edge(tmp_cycle_parameter[cycle_rm_it + 1], tmp_cycle_parameter[cycle_rm_it - 1])
            if neighbor_flag:
                del tmp_cycle_parameter[cycle_rm_it]
                is_removed = True
        if is_removed:
            if is_all_vertex_in_threshold(tmp_cycle_parameter):
                cycle_parameter = tmp_cycle_parameter.copy()
            else:
                tmp_cycle_parameter = cycle_parameter.copy()
    return cycle_parameter



numbed_of_point, threshold = map(int, city_f.readline().split())
for line in city_f :
    p = list(map(int, line.split()))
    if p in point_list:
        print( &quot;invalid point input&quot;)
        sys.exit()
    else : point_list.append(p)

# make base Delauny graph
tri = Delaunay(point_list)
PointArray = np.array(point_list)

Graph = nx.Graph()  # Graph
for i in range(1, numbed_of_point + 1) :
   Graph.add_node(i, pos=point_list[i - 1])

for t in tri.simplices.copy() :
    Graph.add_edge(t[0] + 1, t[1] + 1, color='black', thickness=1, weight=getDistance(t[0], t[1]))
    Graph.add_edge(t[1] + 1, t[2] + 1, color='black', thickness=1, weight=getDistance(t[1], t[2]))
    Graph.add_edge(t[0] + 1, t[2] + 1, color='black', thickness=1, weight=getDistance(t[0], t[2]))

# make all_path_length
path = dict(nx.all_pairs_dijkstra_path_length(Graph))
edgeLengthList = []
for node1 in Graph.node:
    tmp_edge_len = []
    for node2 in Graph.node:
        tmp_edge_len.append(path[node1][node2])
    edgeLengthList.append(tmp_edge_len)

hull = ConvexHull(point_list)
convex = hull.simplices
modiConvex = list(hull.vertices)
modiConvex = [ i+1 for i in modiConvex ]

#find graph center-closed vertex
X , Y = PointArray[:, 0],  PointArray[:, 1]
X_new , Y_new = sum(list(X))/len(X), sum(list(Y))/len(Y)
plt.plot(X_new,Y_new,'k+')

minimum =  math.sqrt((X_new - point_list[0][0]) ** 2 + (Y_new - point_list[0][1]) ** 2)
shortest_candidate_list = []
for a in range(len(point_list)):
    shortest_candidate_list.append([a, math.sqrt((X_new - point_list[a][0]) ** 2 + (Y_new - point_list[a][1]) ** 2)])
shortest_candidate_list = sorted(shortest_candidate_list, key=lambda x: x[1])[0:numbed_of_point]



for iteration in shortest_candidate_list:
    cycle_make_list = []            # before making cycle, check vertex
    cycle_make_from_to = []
    center = iteration[0]
    cycle_make_list.append(center+1)
    print('center is {}'.format(center+1))

    for modiInd in modiConvex:
        lengthSum = 0
        path = nx.shortest_path(Graph, modiInd, center + 1, weight='weight')
        checkBool, checkLength, tmpvertex = True, 0, 0
        for j in range(len(path) - 1):
            length = edgeLengthList[path[j] - 1][path[j + 1] - 1]
            if lengthSum + length &amp;lt;= threshold:
                lengthSum += length
            else:
                checkLength = edgeLengthList[path[j] - 1][path[-1] - 1]
                checkBool = False
                tmpvertex = path[j]
                break

        if checkBool is False:  # neighbor 중에서 W0가 더 작으면서 center와 거리가 더 가까워지는 애가 있으면 선택 (항상 최적은 아님)
            isEnough = True
            while isEnough is True:
                isEnough = False
                neighbor = [i for i in Graph.neighbors(tmpvertex)]
                for ind in neighbor:
                    tmpLength = edgeLengthList[ind - 1][tmpvertex - 1]
                    if tmpLength + lengthSum &amp;lt;= threshold and edgeLengthList[ind - 1][center] &amp;lt;= checkLength:
                        lengthSum = tmpLength + lengthSum
                        tmpvertex = ind
                        checkLength = edgeLengthList[ind - 1][center]
                        isEnough = True
                cycle_make_list.append(tmpvertex)
                cycle_make_from_to.append([modiInd, tmpvertex])
        else:
            cycle_make_from_to.append([modiInd, center + 1])

    #cycle vertex to concaveHull
    cycle_make_list = list(set(cycle_make_list))

    li = [ point_list[i-1] for i in cycle_make_list]
    li = np.array(li)
    concave = cv.concaveHull(li,3)
    concave_vertex = [get_vertex_num(list(i)) + 1 for i in concave]

    # make cycle using shortest path between concaveHaull
    cycle_list, vertex_check_list = create_cycle_from_concave(concave_vertex)
    cycle_list = remove_duplicated_vertex(cycle_list, vertex_check_list)
    cycle_list = optimize_concaved_cycle(cycle_list, concave_vertex, cycle_make_from_to)

    # vertex between cycle and convex &amp;amp; cycle inside
    add_list = []
    for it1 in range(1, len(point_list) + 1):
        if it1 in cycle_list or it1 in modiConvex:          continue
        else:
            cur_vertex , min_dist = 0, 1000 * 1000
            for it2 in cycle_list:
                cur_dist , it1_to_it2_path = 0, nx.shortest_path(Graph,it1,it2,weight='weight')
                for k in range(len(it1_to_it2_path) - 1):
                    cur_dist += edgeLengthList[it1_to_it2_path[k]-1][it1_to_it2_path[k+1]-1]
                if cur_dist &amp;lt; min_dist:
                    min_dist , cur_vertex = cur_dist , it2
            if min_dist &amp;gt; threshold:            # CYCLE 에 가장 가까운 게 threshold 보다 크면 추가해야 됨
                add_list.append(it1)
            else:
                cycle_make_from_to.append([it1,cur_vertex])


    if len(add_list)==0: # all conditions met
        cycle_list.append(cycle_list[0])
        cycle_list = get_optimal_cycle(cycle_list)
        if cycle_list[0] == cycle_list[-1] and len(cycle_list[1:]) == len(set(cycle_list[1:])):
            if is_all_vertex_in_threshold(cycle_list):
                lens = get_cycle_length(cycle_list)
                if lens == cycle_length:
                    print('find a same optimal cycle')
                elif lens &amp;lt; cycle_length:
                    cycle_vertex_list = cycle_list
                    cycle_length = lens
                    final_center = center + 1
                    print(lens)
                    print('new optimal length cycle is {}'.format(cycle_vertex_list))
                else:
                    print('not optimal cycle')
                continue
            else:
                print(&quot;threshold condition not met.&quot;)
        else:
            print(&quot;cycle condition not met&quot;)

    add_vertex_candidate_list = []  # 더 추가할 것이 있을 경우
    for it1 in add_list:
        checkLength, checkVertex, tmpVertex = 0, [], -1
        for it2 in cycle_list:
            lengthSum = 0
            path = nx.shortest_path(Graph, it1, it2, weight='weight')
            for j in range(len(path) - 1):
                length = edgeLengthList[path[j] - 1][path[j + 1] - 1]
                if lengthSum + length &amp;lt; threshold:
                    lengthSum += length
                else:
                    tmpVertex = path[j]
                    break
            if checkLength &amp;lt; lengthSum:
                checkLength = lengthSum
                checkVertex.append(tmpVertex)
        if len(checkVertex) != 0:
            add_vertex_candidate_list.append([it1, checkVertex])
        else:
            add_vertex_candidate_list.append([it1, [it1]])


    add_vertex_candidate = []   # 1개짜리 remove 한 뒤 포함되어 있는 다른 것 다 제거.
    remove_one_List = []
    for index in add_vertex_candidate_list:
        if len(index[1]) == 1:
            add_vertex_candidate.append(index[1][0])
            if index not in remove_one_List:
                remove_one_List.append(index)
            for value in add_vertex_candidate_list:
                if value[0] == index[1][0] and value not in remove_one_List:
                    remove_one_List.append(value)
                for val in value[1]:
                    if val == index[1][0] and value not in remove_one_List:
                        remove_one_List.append(value)

    for i in remove_one_List:
        add_vertex_candidate_list.remove(i)
    can_remove = True
    add_vertex_candidate = list(set(add_vertex_candidate))  # 제거한 것 concave 어디에 위치시킬 지 결정하기
    for add_vertex_iter in add_vertex_candidate:
        result_list = get_inserted_concave(add_vertex_iter, concave_vertex, cycle_make_from_to)
        if result_list == 0:    # 여기서 안 될 경우 뒤에서 되는것 고려
            continue
        concave_vertex = result_list[0]

    check_bool = True
    result_zero = False
    while check_bool:
        check_count = []
        count_List = []
        for index in add_vertex_candidate_list:
            for candidate_iter in index[1]:
                if candidate_iter not in count_List:
                    count_List.append(candidate_iter)
                    check_count.append([candidate_iter,[index[0]]])
                else:
                    for i in check_count:
                        if i[0] == candidate_iter:
                            i[1].append(index[0])
        comp , target_vertex  = 1 , 0

        for it in check_count:
            if comp &amp;lt; len(it[1]):
                target_vertex, comp = it[0], len(it[1])

        if target_vertex != 0:  #하나 이상의 vertex가 겹치면
            result_list = get_inserted_concave(target_vertex, concave_vertex, cycle_make_from_to)
            if result_list ==0:
                result_zero = True
                break
            concave_vertex = result_list[0]
            cycle_list = result_list[1]
            remove_one_List = []
            for value in add_vertex_candidate_list:      # 이때 추가한 target 을 포함하고 있는 add_vertex_list 제거해주기  위에 함
                for val in value[1]:
                    if val == target_vertex and value not in remove_one_List:
                        remove_one_List.append(value)
                if value[0] == target_vertex:
                    remove_one_List.append(value)
            for i in remove_one_List:
                add_vertex_candidate_list.remove(i)
        else:
            check_bool = False

    if result_zero:
        print(&quot;threshold condition not met.&quot;)
        continue

    for ind in add_vertex_candidate:
        for ind2 in add_vertex_candidate_list:
            if ind in ind2[1]:
                add_vertex_candidate.append(ind2[0])
                add_vertex_candidate_list.remove(ind2)

    #1개 짜리 확인용 cycle 생성
    cycle_list.clear()
    for it in range(len(concave_vertex) - 1):
        start, end = concave_vertex[it], concave_vertex[it + 1]
        path_list = nx.shortest_path(Graph, start, end, weight='weight')
        for it2 in range(len(path_list) - 1):
            cycle_list.append(path_list[it2])
    start, end = concave_vertex[-1] , concave_vertex[0]
    if start != end:
        path_list = nx.shortest_path(Graph, start, end, weight='weight')
        for iter2 in range(len(path_list) - 1):
            cycle_list.append(path_list[iter2])

    # 1개짜리 combination
    exist_flag = False
    append_list = []
    for it in add_vertex_candidate_list:
        exist_flag = False
        for it2 in it[1]:
            if it2 in cycle_list:
                exist_flag = True
                break
        if not exist_flag: append_list.append(it[1])

    if len(append_list) != 0:                           # 2개 이상일 때
        final_cycle_candidate = []
        if len(append_list) == 1:                             #1개 인 경우 combination 없이 두개 비교해서 답
            for add_iter in append_list[0]:
                temp_cycle_candidate = []
                result_list = get_inserted_concave(add_iter, concave_vertex, cycle_make_from_to)
                if result_list == 0:
                    continue
                final_cycle_candidate.append(result_list)
        else :
            append_list = list(product(*append_list))               # 여러 개인 경우 combination 들을 비교,
            append_list = [list(i) for i in append_list]
            final_cycle_candidate = []
            for list_val in append_list:
                val_1, val_2 = list_val[0] , list_val[1]
                tmp_final_cycle_candidate = []

                for concave_index_1 in range(len(concave_vertex)):
                    for concave_index_2 in range(len(concave_vertex)):
                        concave_tmp = concave_vertex.copy()
                        concave_tmp.insert(concave_index_1, val_1)
                        concave_tmp.insert(concave_index_2, val_2)
                        tem_cycle, tem_vertex = create_cycle_from_concave(concave_tmp)
                        tem_cycle = remove_duplicated_vertex(tem_cycle, tem_vertex)
                        tem_cycle = optimize_concaved_cycle(tem_cycle, tem_vertex, cycle_make_from_to)
                        tem_cycle.append(tem_cycle[0])
                        tem_cycle_length = get_cycle_length(tem_cycle)
                        if tem_cycle[0] == tem_cycle[-1] and len(tem_cycle[1:]) == len(set(tem_cycle[1:])):
                            tmp_final_cycle_candidate.append([concave_tmp, tem_cycle, tem_cycle_length])

                for concave_index_3 in range(len(concave_vertex)):
                    concave_tmp = concave_vertex.copy()
                    concave_tmp.insert(concave_index_3, val_2)
                    concave_tmp.insert(concave_index_3, val_1)
                    tem_cycle, tem_vertex = create_cycle_from_concave(concave_tmp)
                    tem_cycle = remove_duplicated_vertex(tem_cycle, tem_vertex)
                    tem_cycle = optimize_concaved_cycle(tem_cycle, tem_vertex, cycle_make_from_to)
                    tem_cycle.append(tem_cycle[0])
                    tem_cycle_length = get_cycle_length(tem_cycle)
                    if tem_cycle[0] == tem_cycle[-1] and len(tem_cycle[1:]) == len(set(tem_cycle[1:])):
                        tmp_final_cycle_candidate.append([concave_tmp, tem_cycle, tem_cycle_length])

                if len(tmp_final_cycle_candidate) != 0:
                    tmp_result_list = sorted(tmp_final_cycle_candidate, key=lambda x: x[2])[0]
                    final_cycle_candidate.append(tmp_result_list)

        if len(final_cycle_candidate) != 0:
            cycle_data = sorted(final_cycle_candidate, key=lambda x: x[2])[0]
            concave_vertex = cycle_data[0]
            cycle_list = cycle_data[1]

    final_list = []                     # 마지막으로 추가 안된 vertex 한번 더 추가
    for it1 in range(1, len(point_list) + 1):
        if it1 not in cycle_list:
            cur_vertex, min_dist = 0, 1000 * 1000
            for it2 in cycle_list:
                cur_dist, it1_to_it2_path = 0, nx.shortest_path(Graph, it1, it2, weight='weight')
                for k in range(len(it1_to_it2_path) - 1):
                    cur_dist += edgeLengthList[it1_to_it2_path[k] - 1][it1_to_it2_path[k + 1] - 1]
                if cur_dist &amp;lt; min_dist:
                    min_dist, cur_vertex = cur_dist, it2
            if min_dist &amp;gt; threshold:  # CYCLE 에 가장 가까운 게 threshold 보다 크면 추가해야 됨
                final_list.append(it1)

    result_zero = False
    if len(final_list) != 0:
        make_cycle_possible = True
        for final_index in final_list:
            if final_index in concave_vertex: continue
            can_go_cycle = False
            for cycle_ind in cycle_list:
                final_to_cycle_length = nx.shortest_path_length(Graph,final_index,cycle_ind,weight='weight')
                if final_to_cycle_length &amp;lt;= threshold:
                    can_go_cycle = True
                    break
            if can_go_cycle: continue

            cur_vertex, tmp_cycle_candidate = [] , []
            for cycle_index in cycle_list:
                cur_dist, final_to_cycle_path = 0, nx.shortest_path(Graph, final_index, cycle_index, weight='weight')
                for k in range(len(final_to_cycle_path) - 1):
                    length = edgeLengthList[final_to_cycle_path[k] - 1][final_to_cycle_path[k + 1] - 1]
                    if cur_dist + length &amp;lt; threshold:
                        cur_dist += length
                        if final_to_cycle_path[k+1] not in cur_vertex:
                            cur_vertex.append(final_to_cycle_path[k+1])
                    else:   break

            cur_vertex = list(set(cur_vertex))  # 갈수 있는 vertex 후보
            for vertex_value in cur_vertex:
                tmp_concave_vertex_list = concave_vertex.copy()
                result = get_inserted_concave(vertex_value, tmp_concave_vertex_list, cycle_make_from_to)
                if result == 0:
                    continue
                tmp_cycle_candidate.append(result)

            if len(tmp_cycle_candidate) != 0:
                tmp_cycle_data = sorted(tmp_cycle_candidate, key=lambda x: x[2])[0]
                concave_vertex = tmp_cycle_data[0]
                cycle_list = tmp_cycle_data[1]
            else:
                make_cycle_possible = False
                break
        if not make_cycle_possible:
            print(&quot;can't make cycle&quot;)
            continue

        cycle_list = get_optimal_cycle(cycle_list)
        if cycle_list[0] == cycle_list[-1] and len(cycle_list[1:]) == len(set(cycle_list[1:])):
            if is_all_vertex_in_threshold(cycle_list):
                lens = get_cycle_length(cycle_list)
                if lens == cycle_length:
                    print('find a same optimal cycle')
                elif lens &amp;lt; cycle_length:
                    cycle_vertex_list = cycle_list
                    cycle_length = lens
                    final_center = center + 1
                    print(lens)
                    print('new optimal length cycle is {}'.format(cycle_vertex_list))
                else:
                    print('not optimal cycle')
                continue
            else:
                print(&quot;threshold condition not met.&quot;)
        else:
            print(&quot;cycle condition not met&quot;)
    else:
        if len(append_list) == 0:            # append_list==0 이면 cycle 다시 making 해야함
            tmp_cycle, tmp_vertex = create_cycle_from_concave(concave_vertex)
            tmp_cycle = remove_duplicated_vertex(tmp_cycle, tmp_vertex)
            tmp_cycle = optimize_concaved_cycle(tmp_cycle, concave_vertex, cycle_make_from_to)
            tmp_cycle.append(tmp_cycle[0])
            cycle_list = tmp_cycle

        cycle_list = get_optimal_cycle(cycle_list)
        if cycle_list[0] == cycle_list[-1] and len(cycle_list[1:]) == len(set(cycle_list[1:])):
            if is_all_vertex_in_threshold(cycle_list):
                if lens == cycle_length:
                    print('find a same optimal cycle')
                elif lens &amp;lt; cycle_length:
                    cycle_vertex_list = cycle_list
                    cycle_length = lens
                    final_center = center + 1
                    print(lens)
                    print('new optimal length cycle is {}'.format(cycle_vertex_list))
                else:
                    print('not optimal cycle')
                continue
            else:
                print(&quot;threshold condition not met.&quot;)
        else:
            print(&quot;cycle condition not met&quot;)

for vertex_index in range(len(cycle_vertex_list)-1):
    start, end = cycle_vertex_list[vertex_index], cycle_vertex_list[vertex_index+1]
    edge_len = edgeLengthList[start - 1][end - 1]
    Graph.add_edge(start, end, color='r', weight=edge_len, thickness=5)


print('optimal cycle center is {}'.format(final_center))
print('optimal cycle length is {}'.format(len(cycle_vertex_list)))
print('optimal cycle is {}'.format(cycle_vertex_list))

cycle_f.writelines('{}\n'.format((len(cycle_vertex_list))))
for i in cycle_vertex_list:
    cycle_f.write('{} '.format(i))
cycle_f.close()


# Graph Setting
edge_labels = dict([((u, v,), d['weight']) for u, v, d in Graph.edges(data=True)])
pos=nx.get_node_attributes(Graph, 'pos')
edges = Graph.edges()
colors = [Graph[u][v]['color'] for u, v in edges]
thicknesses = [Graph[u][v]['thickness'] for u, v in edges]
nx.draw_networkx_nodes (Graph, pos, node_size=100, alpha=0.7, node_color='skyblue')
nx.draw_networkx_edges (Graph, pos, width=thicknesses, edge_color = colors)
nx.draw_networkx_labels(Graph, pos, font_size=8, font_weight='bold')    # vertex number
nx.draw_networkx_edge_labels(Graph, pos, edge_labels=edge_labels)   #edge length    if want to see clear image, remove this line

plt.tight_layout()
plt.subplots_adjust(left = 0, bottom = 0, right = 1, top = 1, hspace = 0, wspace = 0)
plt.show()

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/65012667-26530a00-d953-11e9-8198-76a3392c89a8.png&quot; alt=&quot;1&quot; /&gt;
&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/65012676-34088f80-d953-11e9-9754-50845c8ccd54.png&quot; alt=&quot;2&quot; /&gt;
&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/65012694-42ef4200-d953-11e9-8044-d4f5db742376.png&quot; alt=&quot;3&quot; /&gt;&lt;/p&gt;
</description>
        <pubDate>Tue, 17 Sep 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/graph/algoithm/np-complete/delaunay/2019/09/17/Applied-Graph-Theory-Mid-Term-Review/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/graph/algoithm/np-complete/delaunay/2019/09/17/Applied-Graph-Theory-Mid-Term-Review/</guid>
        
        <category>graph</category>
        
        <category>algorithm</category>
        
        <category>NP-complete</category>
        
        <category>Tool</category>
        
        <category>programming</category>
        
        
        <category>graph</category>
        
        <category>algoithm</category>
        
        <category>NP-complete</category>
        
        <category>Delaunay</category>
        
      </item>
    
      <item>
        <title>[문자열] 접미사배열(Suffix Array)과 LCP(Longest Common Prefix), Counting Sort 정리</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#reference&quot; id=&quot;markdown-toc-reference&quot;&gt;Reference&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Suffix Array(접미사 배열)은 어떤 문자열의 suffix(접미사)들을 사전순으로 나열한 배열을 의미한다. 특정문자열에서 반복적으로 나타나는 부분문자열을 찾는 문제에 자주 쓰이는 방법이다. 예를들어 문자열 &lt;strong&gt;ATTCTCG&lt;/strong&gt;가 있을경우
접미사 배열은 다음과 같이 만들어 진다. 그림을 보면 문자열 앞에서부터 한칸씩 나가면서 문자열을 잘라낸다.
그리고 이렇게 잘라낸 문자열을 정렬하고 난 뒤 LCP(Longest Common Prefix) 를 구하면 된다. LCP[i]의 값은 Suffix Array에서 &lt;strong&gt;i&lt;/strong&gt;번째 접미사와 &lt;strong&gt;i-1&lt;/strong&gt;번째 접미사의 가장 긴 공통 접두사의 길이이다. 여기서 LCP의 값이 가장 큰 것이 바로 해당 문자열에서 가장 긴 반복 부분문자열의 길이이다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452355-50efba80-4798-11e9-8c73-758b825cb1ff.png&quot; alt=&quot;1552623515645&quot; /&gt;&lt;/p&gt;

&lt;p&gt;일반화하여 문자열 길이가 N일 때, N개의 접미사를 다 생성한 후 정렬할 때 한 번의 비교에 O(N)의 시간이 걸리고 접미사는 N개이므로 일반적인 정렬 반복횟수는 O(NlogN)이 되어, 전체적인 시간복잡도는 다음과 같다.&lt;/p&gt;

&lt;script type=&quot;math/tex; mode=display&quot;&gt;O(N^2 logN)&lt;/script&gt;

&lt;p&gt;위 알고리즘의 경우 N 이 커질 경우 사용할 수 없게 된다.  이를 해결하기 위해 다른 방법을 생각해보자. d=1부터 시작해서, **매번 접미사의 앞 d * 2글자만으로 정렬한다. 매 루프가 지날 때마다 d는 2배가 된다. **즉 d가 1, 2, 4, 8, … 식으로 늘어나 정렬을 할 경우 O(logN) 번째 걸리는 것을 알 수 있다. 전체 정렬에 O(NlogN)이 걸린다면 전체 시간복잡도는 다음과 같다.&lt;/p&gt;

&lt;script type=&quot;math/tex; mode=display&quot;&gt;O(N(logN)^2)&lt;/script&gt;

&lt;p&gt;ATTCTCG를 길이 1 단위로 그룹을 지어보면 다음과 같다. 사전순으로 String에 그룹숫자를 부여한다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452408-6ebd1f80-4798-11e9-8b55-4849cb88ab91.png&quot; alt=&quot;1552638307906&quot; /&gt;&lt;/p&gt;

&lt;p&gt;이후 길이 d=1 단위로 본다는 것은, string[0..1], string[1..2] , string[2..3], .. 값들을 비교하는 것이다. 아래 그림에서 길이 2 * 1의 string을 비교하여 그룹숫자를 부여하면 다음과 같다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452424-7bda0e80-4798-11e9-8107-06fc905e0c00.png&quot; alt=&quot;1552638328262&quot; /&gt;&lt;/p&gt;

&lt;p&gt;결과에서 보면 청록색 부분의 string, g[3]과 g[5]이 같은 그룹인덱스인 것을 확인할 수 있다. 
&lt;strong&gt;여기서 3 길이 단위를 구하지 않고,  4길이 단위를 바로 구할수 있다. 아래 그림을 보자.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452442-87c5d080-4798-11e9-9617-47f3219d054b.png&quot; alt=&quot;1552638484924&quot; /&gt;&lt;/p&gt;

&lt;p&gt;위 d=1 에서 구분짓지 못했던 g[3]과 g[5]를 확인하려면 연두색 부분을 보면 된다. 이때 T와 G, C와 *를 순서대로 비교하는 것은 효율적이지 못하다. 우리는 이미 d=1 값을 구하면서 연두색값에 해당하는 그룹인덱스를 구한것이 주황색임을 알수있다. 즉,  t개 길이 단위에서 g값이 결정되어 있을때 아래식을 비교하면 된다.&lt;/p&gt;

&lt;script type=&quot;math/tex; mode=display&quot;&gt;if (g[x] == g[y]) \rightarrow compare \space g[x+t] \space , \space g[y+t]&lt;/script&gt;

&lt;p&gt;그러므로 2개 길이 단위에서 4개길이 단위를 만드는데, 최대 비교회수는 2번이다.  간단히 얘기하자면, 코딩할 때 (g[x],g[x+t])와 (g[y],g[y+t])를 pair&amp;lt;int,int&amp;gt;로 만들어 놓고 정렬하면 된다는 것이다.  같은 방식으로 [4개 단위에 대한 그룹인덱스]를 가지고 있다면 [8개 단위의 새로운 그룹인덱스]를 만들때에도 최대 2번까지만 비교하게 된다.&lt;/p&gt;

&lt;p&gt;만약 Counting sort로 선택할 경우 정렬 반복횟수는 O(N)이 되어, 전체적인 시간복잡도는 다음과 같다.&lt;/p&gt;

&lt;script type=&quot;math/tex; mode=display&quot;&gt;O(NlogN)&lt;/script&gt;

&lt;p&gt;Counting Sort는 위에서든 예시처럼 정렬하는 String이 특정한 범위(숫자범위, 혹은 알파벳)안에 있을 때 유용하게사용합니다.  문자열 ATTCTCG가 있을 때 counting sort는 말 그대로 각 문자의 갯수를 counting 합니다. 그리하여 얻은 counting 배열을 Accumulated 하여 Accumulated Array를 생성합니다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452466-9a400a00-4798-11e9-88b0-7579e6cd1485.png&quot; alt=&quot;1552663505847&quot; /&gt;&lt;/p&gt;

&lt;p&gt;이후 문자열 끝부터 (1) 시작하여 Accumulated Array에 해당하는 값의 인덱스값을 체크합니다(2). 그리고 빈 Array에 해당 값을 복사한 뒤(3) 인덱스값을 한칸 앞으로 당긴 뒤(4), 이와같은 과정을 반복하면 N번만에 정렬이 됩니다. 이렇게 빠른 속도로 정렬이 가능하지만  대부분 상황에서 많은 메모리 낭비하는 단점이 있습니다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452492-a7f58f80-4798-11e9-8d72-67ffca2193f8.png&quot; alt=&quot;1552663556322&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452521-b93e9c00-4798-11e9-8698-7a95c9be3daa.png&quot; alt=&quot;1552665326435&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Counting Sort for stable sorting&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;typedef pair&amp;lt;int,int&amp;gt; pii;
vector&amp;lt;pii&amp;gt; sort(vector&amp;lt;pii&amp;gt; &amp;amp;v) {
    int n=v.size();
    vector&amp;lt;pii&amp;gt; ans(n);
    vector&amp;lt;int&amp;gt; cnt(n+1,0);
    vector&amp;lt;int&amp;gt; idx(n,0);
 
    for(int i=0; i&amp;lt;n; i++) cnt[v[i].second]++;
    for(int i=1; i&amp;lt;=n; i++) cnt[i]+=cnt[i-1];
    for(int i=n-1; i&amp;gt;=0; i--) idx[--cnt[v[i].second]]=i;
 
    cnt.clear(); cnt.resize(n+1,0);
    for(int i=0; i&amp;lt;n; i++) cnt[v[i].first]++;
    for(int i=1; i&amp;lt;=n; i++) cnt[i]+=cnt[i-1];
    for(int i=n-1; i&amp;gt;=0; i--) ans[--cnt[v[idx[i]].first]]=v[idx[i]];
 
    return ans;
}

12: cnt배열을 이용해서 v[i].second의 개수를 세준다.
13: 값을 누적시켜준다.
14: idx값은 second 기준으로 정렬했을 때 값이 아닌 인덱스를 저장한다.
만약 second를 기준으로만 정렬하고 말거라면, ans[--cnt[v[i].second]]=v[i].second;일 것이다.
그런데 우리는 값이 아닌 원래 정렬되기 전 상태의 인덱스가 필요하므로 ans[--cnt[v[i].second]]=i;라고 저장한 것이며,이게 사실 답은 아니니까 ans[]배열이 아닌 idx[]배열에 저장한 것 뿐이다.
그럼 14번째 줄까지 하고 for(int i=0; i&amp;lt;n; i++) printf(&quot;%d &quot;, idx[i]);를 실행하게 되면,
second기준으로 정렬했을 때 v[]배열의 인덱스를 순서대로 출력하는 것이다. 즉, 이 순서대로 first에 접근하면 second크기 순으로 first에 접근할 수 있으며, 다시 이전과 같은 방법으로 first를 정렬하면 second순서를 이전과 같이 최대한 유지하면서 first를 정렬할 수 있다.

출처: https://plzrun.tistory.com/entry/Counting-Sort-Radix-Sort [plzrun's algorithm]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;아래 코드는 두 sequence의 Longest Common Substring를 찾기 위해 Suffix Array와 Longest Common Prefix를 적용한 코드입니다. 파이썬으로 작성되었고,  시간복잡도는 O(Nlog^2N) 입니다.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# Find Longest Common Prefix using suffix array
# first, construct 2 suffix array           -   O(Nlog^N + Mlog^M)
# second,  merge 2 suffix array             -   O(N(a) + M(b))     
a , b  = average of Suffix length compared to each other
# second, find LCP                          -

import time
import math

def suffix_array(txt):
    if not txt:
        return []
    txt += chr(0)
    N = len(txt)
    #print &quot;input string = [&quot;, txt, &quot;]&quot;

    equivalence = {t: i for i, t in enumerate(sorted(set(txt)))}
    #print &quot;equi=&quot;, equivalence                          #bucket number set
    cls = [equivalence[t] for t in txt]
    #print &quot;cls=&quot;, cls                                   # first index sorting

    ns = [(2**i)%N for i in range( int( math.ceil( math.log(N,2))))]
    #print &quot;ns=&quot;, ns

    for n in ns[:-1]:
        result = sorted(zip(cls, cls[n:]+cls[:n], range(N)))
        result0, result1, inds = list(zip(*result))

        cls[inds[0]] = 0
        for j in range(1, N):
            cls[inds[j]] = cls[inds[j-1]]
            if (result0[j], result1[j]) != (result0[j-1], result1[j-1]):
                cls[inds[j]] += 1
    n = ns[-1]
    result = sorted(zip(cls, cls[n:]+cls[:n], range(N)))
    return list(list(zip(*result))[2][1:])

def merge(sufarr1, sufarr2 , sufArrIndex1 , sufArrIndex2):      # sort(sumLa)  &amp;lt;&amp;lt; time check need
    indMerge = []
    i, j = 0, 0
    len1, len2 = len(sufarr1), len(sufarr2)
    while True:
        if i == len1 and j == len2:             break
        if i == len1:           # if finished sufarr1
            indMerge.append(-sufArrIndex2[j]-1)
            j += 1
        elif j == len2:         # if finished sufarr2
            indMerge.append(sufArrIndex1[i]+1)
            i += 1
        else:
            index1, index2 = sufArrIndex1[i], sufArrIndex2[j]
            str1 , str2 = sufarr1[index1:] , sufarr2[index2:]
            if str1 &amp;lt; str2 :              # standard character-by-character comparison rules
                indMerge.append(index1+1)
                i += 1
            elif str1 &amp;gt; str2:
                indMerge.append(-index2-1)
                j += 1
            else:
                indMerge.append(index1+1)
                indMerge.append(-index2-1)
                i += 1
                j += 1
    return indMerge

def getLCP(suffix1, suffix2, indMerge):
    lcp, where = [0] , []
    length = len(indMerge)
    position = 0 if indMerge[0] &amp;gt; 0 else 1
    where.append(position)
    for x in range(length-1):
        index = indMerge[x]
        if indMerge[x] &amp;gt; 0:
            str1 = suffix1[index - 1:]
        else:
            str1 = suffix2[-index-1:]
        index = indMerge[x+1]
        if indMerge[x+1] &amp;gt; 0:
            str2 = suffix1[index - 1:]
            where.append(0)
        else:
            str2 = suffix2[-index-1:]
            where.append(1)
        size = min(len(str1), len(str2))
        count = 0
        for y in range(size):
            if str1[y] == str2[y]:
                count += 1
            else:
                break
        lcp.append(count)
    return lcp, where

def lcped(suffixMerge,seq1,seq2):
    rank , lcp = [] , [0]
    N = len(suffixMerge)
    for i in range(N):
        rank.insert(suffixMerge[i],i)
    length = 0
    for i in range(N):
        k = rank[i]
        if k:
            j = suffixMerge[k-1]
            while True:
                if j == N :
                    break
                if j+length &amp;lt; N / 2:
                    str1 = seq1[j + length]
                elif j + length &amp;lt; N:
                    str1 = seq2[j + length - N / 2]
                else:
                    break
                if i+length &amp;lt; N / 2:
                    str2 = seq1[i + length]
                elif i + length &amp;lt; N:
                        str2 = seq2[i + length - N / 2]
                else:
                    break
                #str1 = seq1[j + length] if j &amp;lt; N / 2 else seq2[j + length - N / 2]
                #str2 = seq1[i + length] if i &amp;lt; N / 2 else seq2[i + length - N / 2]
                if str1 == str2:
                    length+=1
                else:
                    break
            lcp.insert(k,length)
            if length:
                length-=1
        i+=1
    return lcp

def readSequnce(sequence):
    f = open(sequence)
    for line in f:
        seq = line
    f.close()
    return seq

def main(sequence1,sequence2):
    seq1 = readSequnce(sequence1)
    seq2 = readSequnce(sequence2)
    start_time = time.time()
    la1= suffix_array(seq1)
    la2= suffix_array(seq2)
    print len(la1) , len(la2)
    suffixMerge = merge(seq1,seq2,la1,la2)
    lcp, position = getLCP(seq1,seq2,suffixMerge)
    maxValue, maxStr, compare = lcp[0],'',position[0]
    N = len(suffixMerge)
    for i in range(N):
        if i==0: continue
        if lcp[i] &amp;gt; maxValue and position[i] != position[i-1]:
            maxValue = lcp[i]
            maxStr = seq1[suffixMerge[i]-1:suffixMerge[i]+maxValue-1] if suffixMerge[i] &amp;gt; 0 else seq2[-suffixMerge[i]-1: -suffixMerge[i] + maxValue-1]
    print(&quot;--- %s seconds ---&quot; % (time.time() - start_time))
    print maxStr, maxValue

if __name__ == '__main__':
    main('seq819200_1.txt','seq819200_2.txt')

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452556-ca87a880-4798-11e9-9b5a-a32bdb57c1d4.png&quot; alt=&quot;1552532578359&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452608-e25f2c80-4798-11e9-8259-547a12a50e90.png&quot; alt=&quot;1552532591551&quot; /&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th style=&quot;text-align: center&quot;&gt;size&lt;/th&gt;
      &lt;th style=&quot;text-align: center&quot;&gt;0.1M&lt;/th&gt;
      &lt;th style=&quot;text-align: center&quot;&gt;0.2M&lt;/th&gt;
      &lt;th style=&quot;text-align: center&quot;&gt;0.4M&lt;/th&gt;
      &lt;th style=&quot;text-align: center&quot;&gt;0.8M&lt;/th&gt;
      &lt;th style=&quot;text-align: center&quot;&gt;1.6M&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;time(s)&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;132.77s&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;549.5s&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;2444.29s&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;9689.48s&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;31983.4s&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;LCS&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;26&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;80&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;119&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;177&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;170&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;input 사이즈 크기에 따른 LCS 길이의 차이를 볼 수 있습니다. 위 LCS 는 실제 sequencing data를 사용하였으며,
아래 blast 툴을 통해 확인 해 볼 수 있습니다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54452632-f440cf80-4798-11e9-98d3-5554490043ce.png&quot; alt=&quot;그림입니다&quot; /&gt;&lt;/p&gt;

&lt;h4 id=&quot;reference&quot;&gt;Reference&lt;/h4&gt;

&lt;p&gt;https://blog.myungwoo.kr/57&lt;/p&gt;

&lt;p&gt;https://bowbowbow.tistory.com/8&lt;/p&gt;

&lt;p&gt;https://plzrun.tistory.com/entry/Suffix-Array-ONlogNlgN%EA%B3%BC-ONlogN-%EA%B5%AC%ED%98%84-%EB%B0%8F-%EC%84%A4%EB%AA%85&lt;/p&gt;
</description>
        <pubDate>Sat, 16 Mar 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/suffix%20array/python/lcp/lcs/counting%20sort/2019/03/16/Suffix-Array-%EB%B0%8F-Counting-Sort-%EC%A0%95%EB%A6%AC/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/suffix%20array/python/lcp/lcs/counting%20sort/2019/03/16/Suffix-Array-%EB%B0%8F-Counting-Sort-%EC%A0%95%EB%A6%AC/</guid>
        
        <category>suffix array</category>
        
        <category>python</category>
        
        <category>lcp</category>
        
        <category>lcs</category>
        
        <category>counting sort</category>
        
        
        <category>suffix array</category>
        
        <category>python</category>
        
        <category>lcp</category>
        
        <category>lcs</category>
        
        <category>counting sort</category>
        
      </item>
    
      <item>
        <title>[Docker] 개념 정리 및 사용방법까지.</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#docker-개념-관리-이미지생성까지-한번에&quot; id=&quot;markdown-toc-docker-개념-관리-이미지생성까지-한번에&quot;&gt;Docker 개념, 관리, 이미지생성까지 한번에!!&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#docker&quot; id=&quot;markdown-toc-docker&quot;&gt;Docker&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#컨테이너란-무엇인가&quot; id=&quot;markdown-toc-컨테이너란-무엇인가&quot;&gt;컨테이너란 무엇인가?&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#이미지image&quot; id=&quot;markdown-toc-이미지image&quot;&gt;이미지(Image)&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#레이어-저장방식&quot; id=&quot;markdown-toc-레이어-저장방식&quot;&gt;레이어 저장방식&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#이미지-경로&quot; id=&quot;markdown-toc-이미지-경로&quot;&gt;이미지 경로&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#dockerfile&quot; id=&quot;markdown-toc-dockerfile&quot;&gt;Dockerfile&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#도커-설치하기&quot; id=&quot;markdown-toc-도커-설치하기&quot;&gt;도커 설치하기&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#sudo-없이-사용하기&quot; id=&quot;markdown-toc-sudo-없이-사용하기&quot;&gt;sudo 없이 사용하기**&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#컨테이너-실행하기&quot; id=&quot;markdown-toc-컨테이너-실행하기&quot;&gt;컨테이너 실행하기&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#도커-기본-명령어&quot; id=&quot;markdown-toc-도커-기본-명령어&quot;&gt;도커 기본 명령어&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#컨테이너-목록-확인하기-ps&quot; id=&quot;markdown-toc-컨테이너-목록-확인하기-ps&quot;&gt;컨테이너 목록 확인하기 (ps)&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#컨테이너-중지하기-stop&quot; id=&quot;markdown-toc-컨테이너-중지하기-stop&quot;&gt;컨테이너 중지하기 (stop)&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#컨테이너-제거하기-rm&quot; id=&quot;markdown-toc-컨테이너-제거하기-rm&quot;&gt;컨테이너 제거하기 (rm)&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#이미지-목록-확인하기-images&quot; id=&quot;markdown-toc-이미지-목록-확인하기-images&quot;&gt;이미지 목록 확인하기 (images)&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#이미지-다운로드하기-pull&quot; id=&quot;markdown-toc-이미지-다운로드하기-pull&quot;&gt;이미지 다운로드하기 (pull)&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#이미지-삭제하기-rmi&quot; id=&quot;markdown-toc-이미지-삭제하기-rmi&quot;&gt;이미지 삭제하기 (rmi)&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#컨테이너-로그-보기-logs&quot; id=&quot;markdown-toc-컨테이너-로그-보기-logs&quot;&gt;컨테이너 로그 보기 (logs)&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#컨테이너-명령어-실행하기-exec&quot; id=&quot;markdown-toc-컨테이너-명령어-실행하기-exec&quot;&gt;컨테이너 명령어 실행하기 (exec)&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#도커-이미지-만들기&quot; id=&quot;markdown-toc-도커-이미지-만들기&quot;&gt;도커 이미지 만들기&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#docker-컨테이너container-커밋commit하기&quot; id=&quot;markdown-toc-docker-컨테이너container-커밋commit하기&quot;&gt;Docker 컨테이너(Container) 커밋(Commit)하기&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#docker-이미지image-push하기&quot; id=&quot;markdown-toc-docker-이미지image-push하기&quot;&gt;Docker 이미지(Image) Push하기&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#dockerfile-1&quot; id=&quot;markdown-toc-dockerfile-1&quot;&gt;DockerFile&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#dockerfile-기본-명령어&quot; id=&quot;markdown-toc-dockerfile-기본-명령어&quot;&gt;DockerFile 기본 명령어&lt;/a&gt;            &lt;ul&gt;
              &lt;li&gt;&lt;a href=&quot;#1-from&quot; id=&quot;markdown-toc-1-from&quot;&gt;1. FROM&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#2-maintainer&quot; id=&quot;markdown-toc-2-maintainer&quot;&gt;2. MAINTAINER&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#3-copy&quot; id=&quot;markdown-toc-3-copy&quot;&gt;3. COPY&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#4-add&quot; id=&quot;markdown-toc-4-add&quot;&gt;4. ADD&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#5-run&quot; id=&quot;markdown-toc-5-run&quot;&gt;5. RUN&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#6-cmd&quot; id=&quot;markdown-toc-6-cmd&quot;&gt;6. CMD&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#7-workdir&quot; id=&quot;markdown-toc-7-workdir&quot;&gt;7. WORKDIR&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#8-expose&quot; id=&quot;markdown-toc-8-expose&quot;&gt;8. EXPOSE&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#9-volume&quot; id=&quot;markdown-toc-9-volume&quot;&gt;9. VOLUME&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#10-env&quot; id=&quot;markdown-toc-10-env&quot;&gt;10. ENV&lt;/a&gt;&lt;/li&gt;
            &lt;/ul&gt;
          &lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#docker-빌드하기&quot; id=&quot;markdown-toc-docker-빌드하기&quot;&gt;Docker 빌드하기&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#도커-이미지-리팩토링-및-명령어-최적화&quot; id=&quot;markdown-toc-도커-이미지-리팩토링-및-명령어-최적화&quot;&gt;도커 이미지 리팩토링 및 명령어 최적화&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#컨테이너-업데이트&quot; id=&quot;markdown-toc-컨테이너-업데이트&quot;&gt;컨테이너 업데이트&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#docker-compose&quot; id=&quot;markdown-toc-docker-compose&quot;&gt;Docker Compose&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#설치하기&quot; id=&quot;markdown-toc-설치하기&quot;&gt;설치하기&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#docker-composeyml&quot; id=&quot;markdown-toc-docker-composeyml&quot;&gt;docker-compose.yml&lt;/a&gt;            &lt;ul&gt;
              &lt;li&gt;&lt;a href=&quot;#1-version&quot; id=&quot;markdown-toc-1-version&quot;&gt;1. version&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#2-services&quot; id=&quot;markdown-toc-2-services&quot;&gt;2. services&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#3-image&quot; id=&quot;markdown-toc-3-image&quot;&gt;3. image&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#4-volumes&quot; id=&quot;markdown-toc-4-volumes&quot;&gt;4. volumes&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#5-environment&quot; id=&quot;markdown-toc-5-environment&quot;&gt;5. environment&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#6-build&quot; id=&quot;markdown-toc-6-build&quot;&gt;6. build&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#7-port&quot; id=&quot;markdown-toc-7-port&quot;&gt;7. port&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#8-depends_on&quot; id=&quot;markdown-toc-8-depends_on&quot;&gt;8. depends_on&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#9-links&quot; id=&quot;markdown-toc-9-links&quot;&gt;9. links&lt;/a&gt;&lt;/li&gt;
              &lt;li&gt;&lt;a href=&quot;#10-command&quot; id=&quot;markdown-toc-10-command&quot;&gt;10. command&lt;/a&gt;&lt;/li&gt;
            &lt;/ul&gt;
          &lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#docker-composeyml-관리법&quot; id=&quot;markdown-toc-docker-composeyml-관리법&quot;&gt;docker-compose.yml 관리법&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#이미지-저장소&quot; id=&quot;markdown-toc-이미지-저장소&quot;&gt;이미지 저장소&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#배포하기&quot; id=&quot;markdown-toc-배포하기&quot;&gt;배포하기&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#컨테이너-배포-방식으로&quot; id=&quot;markdown-toc-컨테이너-배포-방식으로&quot;&gt;컨테이너 배포 방식으로&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#참고-자료&quot; id=&quot;markdown-toc-참고-자료&quot;&gt;참고 자료&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;docker-개념-관리-이미지생성까지-한번에&quot;&gt;Docker 개념, 관리, 이미지생성까지 한번에!!&lt;/h1&gt;

&lt;p&gt;&lt;strong&gt;서비스 형태의 플랫폼&lt;/strong&gt;(Platform-as-a-service, PaaS)은 서비스 제공업체가 고객에게 플랫폼을 제공함으로써 고객이 일반적으로 소프트웨어 개발 프로세스에 필요한 인프라를 구축하고 유지할 필요 없이 비즈니스 애플리케이션을 개발, 실행, 관리할 수 있도록 하는 클라우드 컴퓨팅의 시대를 말합니다.&lt;/p&gt;

&lt;p&gt;이미 많은 사람들에게 친숙한 기술용어가 되어버린 Container ,그 시대를 이끌고 있는 대표주자가 바로 &lt;a href=&quot;https://www.docker.com/&quot;&gt;Docker&lt;/a&gt;입니다. 이번 글에서는 Docker의 개념, 관리, 이미지 생성에 대하여 정리하겠습니다.&lt;/p&gt;

&lt;h2 id=&quot;docker&quot;&gt;Docker&lt;/h2&gt;

&lt;p&gt;도커는 &lt;strong&gt;컨테이너 기반의 오픈소스 가상화 플랫폼&lt;/strong&gt;입니다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://subicura.com/assets/article_images/2017-01-19-docker-guide-for-beginners-1/docker-works.png&quot; alt=&quot;도커는 컨테이너를 관리하는 플랫폼&quot; /&gt;&lt;/p&gt;

&lt;p&gt;​	Docker는 컨테이너를 관리하는 플랫폼&lt;/p&gt;

&lt;p&gt;다양한 프로그램, 실행환경을 &lt;strong&gt;컨테이너로 추상화하고 동일한 인터페이스&lt;/strong&gt;를 제공하여 프로그램의 배포 및 관리를 단순하게 해줍니다. 백엔드 프로그램, 데이터베이스 서버, 메시지 큐등 어떤 프로그램도 컨테이너로 추상화할 수 있고 조립PC, AWS, Azure, Google cloud등 어디에서든 실행할 수 있습니다.&lt;/p&gt;

&lt;h2 id=&quot;컨테이너란-무엇인가&quot;&gt;컨테이너란 무엇인가?&lt;/h2&gt;

&lt;p&gt;​    운영체계를 기반으로 만들어진 대부분의 Software는 그 실행을 위하여 OS와 Software가 사용하는 동적 Library에 대하여 의존성을 갖습니다. 즉, Software의 실행을 위해선 OS와 Library를 포함, Software가 필요로 하는 파일 등으로 구성된 실행환경이 필요한데, 하나의 시스템 위에서 둘 이상의 Software를 동시에 실행하려고 한다면 문제가 발생할 수 있습니다. 예를 들어, Software A와 B가 동일한 Library를 사용하지만 서로 다른 버전을 필요로 하는 경우라던지 두 software의 운영 체제가 다를 경우 등 다양한 경우에서 문제가 발생할 수 있습니다. 이런 상황에서가장 간단한 해결책은 두 Software를 위한 시스템을 각각 준비하는 것인데, 시스템을 각각 준비할 경우 비용의 문제가 발생하게 된다(10개의 software일 경우 10개의 시스템이 필요). 이러한 문제점을 효율적으로 해결한 것이 바로 &lt;strong&gt;컨테이너&lt;/strong&gt;입니다.  컨테이너(Container)는 개별 Software의 실행에 필요한 실행환경을 독립적으로 운용할 수 있도록 기반환경 또는 다른 실행환경과의 간섭을 막고 &lt;strong&gt;실행의 독립성을 확보해주는 운영체계 수준의 격리 기술&lt;/strong&gt;을 말합니다. 컨테이너는 애플리케이션을 실제 구동 환경으로부터 &lt;strong&gt;추상화&lt;/strong&gt;할 수 있는 논리 패키징 메커니즘을 제공합니다. 
&lt;img src=&quot;https://www.aberdeen.com/wp-content/uploads/2016/10/shipping-container-b.jpg&quot; alt=&quot;Container&quot; /&gt;&lt;/p&gt;

&lt;p&gt;가상 환경에 익숙하다면 컨테이너를 가상 머신(Virtual Machine)에 비교하여 생각하면 이해하기 쉽습니다. 호스트 운영체제에서 구동되며 그 바탕이 되는 &lt;strong&gt;하드웨어&lt;/strong&gt;에 가상으로 액세스하는 Linux, Windows 등의 게스트 운영체제를 의미합니다. 컨테이너는 가상 머신과 마찬가지로 애플리케이션을 관련 라이브러리 및 종속 항목과 함께 패키지로 묶어 소프트웨어 서비스 구동을 위한 격리 환경을 마련해 줍니다. 그러나 아래에서 살펴보듯 VM과의 유사점은 여기까지입니다. 컨테이너를 사용하면 개발자와 IT 운영팀이 훨씬 작은 단위로 업무를 수행할 수 있으므로 그에 따른 이점도 훨씬 많습니다.&lt;/p&gt;

&lt;p&gt;가상 머신은 하드웨어 스택을 가상화합니다. 컨테이너는 이와 달리 &lt;strong&gt;운영체제 수준&lt;/strong&gt;에서 가상화를 실시하여 다수의 컨테이너를 &lt;strong&gt;OS 커널에서 직접 구동&lt;/strong&gt;합니다. 컨테이너는 훨씬 가볍고 운영체제 커널을 공유하며, 시작이 훨씬 빠르고 운영체제 전체 부팅보다 메모리를 훨씬 적게 차지합니다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://cloud.google.com/images/containers-landing/containers-101.png?authuser=0&amp;amp;hl=ko&quot; alt=&quot;img&quot; /&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th style=&quot;text-align: center&quot;&gt; &lt;/th&gt;
      &lt;th style=&quot;text-align: center&quot;&gt;컨테이너의 이점&lt;/th&gt;
      &lt;th style=&quot;text-align: center&quot;&gt;가상 머신의 이점&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;일관성 있는 런타임 환경&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;&lt;em&gt;O&lt;/em&gt;&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;&lt;em&gt;O&lt;/em&gt;&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;애플리케이션 샌드박스화&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;&lt;em&gt;O&lt;/em&gt;&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;&lt;em&gt;O&lt;/em&gt;&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;디스크 용량 절감&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;&lt;em&gt;O&lt;/em&gt;&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;&lt;em&gt;X&lt;/em&gt;&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;낮은 오버헤드&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;&lt;em&gt;O&lt;/em&gt;&lt;/td&gt;
      &lt;td style=&quot;text-align: center&quot;&gt;&lt;em&gt;X&lt;/em&gt;&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;blockquote&gt;
  &lt;p&gt;Container 기술 자체는, 격리된 실행환경을 제공하기 위한 인프라 기술이긴 하지만, Docker는 이미 그 이름에서 향기를 품고 있듯이, 그 목적부터 Software 또는 Service의 빠르고 효율적인 Shipping에 강하게 집중하고 있다. 때문에 Docker는, 근간이 되는 격리 기술을 넘어 Container와 Image, 네트워크와 서비스, 보안 등 Software의 배포와 생명 주기를 관리할 수 있는 다양한 주변 기능에 초점이 맞추어져 있다고 할 수 있다.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2 id=&quot;이미지image&quot;&gt;이미지(Image)&lt;/h2&gt;

&lt;p&gt;이미지는 &lt;strong&gt;컨테이너 실행에 필요한 파일과 설정값등을 포함하고 있는 것&lt;/strong&gt;으로 상태값을 가지지 않고 변하지 않습니다(Immutable). 컨테이너는 이미지를 실행한 상태라고 볼 수 있고 추가되거나 변하는 값은 컨테이너에 저장됩니다. 같은 이미지에서 여러개의 컨테이너를 생성할 수 있고 컨테이너의 상태가 바뀌거나 컨테이너가 삭제되더라도 이미지는 변하지 않고 그대로 남아있습니다.&lt;/p&gt;

&lt;h4 id=&quot;레이어-저장방식&quot;&gt;레이어 저장방식&lt;/h4&gt;

&lt;p&gt;&lt;img src=&quot;https://subicura.com/assets/article_images/2017-01-19-docker-guide-for-beginners-1/image-layer.png&quot; alt=&quot;Docker Layer&quot; /&gt;											Docker Layer&lt;/p&gt;

&lt;p&gt;도커 이미지는 컨테이너를 실행하기 위한 모든 정보를 가지고 있기 때문에 보통 용량이 수백메가MB에 이릅니다. 처음 이미지를 다운받을 땐 크게 부담이 안되지만 기존 이미지에 파일 하나 추가했다고 수백메가를 다시 다운받는다면 매우 비효율적일 수 밖에 없습니다.&lt;/p&gt;

&lt;p&gt;도커는 이런 문제를 해결하기 위해 &lt;strong&gt;레이어layer&lt;/strong&gt;라는 개념을 사용하고 유니온 파일 시스템을 이용하여 여러개의 레이어를 하나의 파일시스템으로 사용할 수 있게 해줍니다. 이미지는 여러개의 읽기 전용 read only 레이어로 구성되고 파일이 추가되거나 수정되면 새로운 레이어가 생성됩니다. ubuntu 이미지가 &lt;code class=&quot;highlighter-rouge&quot;&gt;A&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;B&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;C&lt;/code&gt;의 집합이라면, ubuntu 이미지를 베이스로 만든 nginx 이미지는 &lt;code class=&quot;highlighter-rouge&quot;&gt;A&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;B&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;C&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;nginx&lt;/code&gt;가 됩니다. webapp 이미지를 nginx 이미지 기반으로 만들었다면 예상대로 &lt;code class=&quot;highlighter-rouge&quot;&gt;A&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;B&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;C&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;nginx&lt;/code&gt; + &lt;code class=&quot;highlighter-rouge&quot;&gt;source&lt;/code&gt; 레이어로 구성됩니다. webapp 소스를 수정하면 &lt;code class=&quot;highlighter-rouge&quot;&gt;A&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;B&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;C&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;nginx&lt;/code&gt; 레이어를 제외한 새로운 &lt;code class=&quot;highlighter-rouge&quot;&gt;source(v2)&lt;/code&gt; 레이어만 다운받으면 되기 때문에 굉장히 효율적으로 이미지를 관리할 수 있습니다.&lt;/p&gt;

&lt;p&gt;컨테이너를 생성할 때도 레이어 방식을 사용하는데 기존의 이미지 레이어 위에 읽기/쓰기read-write 레이어를 추가합니다. 이미지 레이어를 그대로 사용하면서 컨테이너가 실행중에 생성하는 파일이나 변경된 내용은 읽기/쓰기 레이어에 저장되므로 여러개의 컨테이너를 생성해도 최소한의 용량만 사용합니다.&lt;/p&gt;

&lt;h4 id=&quot;이미지-경로&quot;&gt;이미지 경로&lt;/h4&gt;

&lt;p&gt;&lt;img src=&quot;https://subicura.com/assets/article_images/2017-01-19-docker-guide-for-beginners-1/image-url.png&quot; alt=&quot;Docker image url&quot; /&gt;&lt;/p&gt;

&lt;p&gt;이미지는 url 방식으로 관리하며 태그를 붙일 수 있습니다. ubuntu 14.04 이미지는 &lt;code class=&quot;highlighter-rouge&quot;&gt;docker.io/library/ubuntu:14.04&lt;/code&gt; 또는 &lt;code class=&quot;highlighter-rouge&quot;&gt;docker.io/library/ubuntu:trusty&lt;/code&gt; 이고 &lt;code class=&quot;highlighter-rouge&quot;&gt;docker.io/library&lt;/code&gt;는 생략가능하여 &lt;code class=&quot;highlighter-rouge&quot;&gt;ubuntu:14.04&lt;/code&gt; 로 사용할 수 있습니다. 이러한 방식은 이해하기 쉽고 편리하게 사용할 수 있으며 태그 기능을 잘 이용하면 테스트나 롤백도 쉽게 할 수 있습니다.&lt;/p&gt;

&lt;h4 id=&quot;dockerfile&quot;&gt;Dockerfile&lt;/h4&gt;

&lt;div class=&quot;language-dockerfile highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;# vertx/vertx3 debian version&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;FROM&lt;/span&gt;&lt;span class=&quot;s&quot;&gt; subicura/vertx3:3.3.1&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;MAINTAINER&lt;/span&gt;&lt;span class=&quot;s&quot;&gt; chungsub.kim@purpleworks.co.kr&lt;/span&gt;


&lt;span class=&quot;k&quot;&gt;ADD&lt;/span&gt;&lt;span class=&quot;s&quot;&gt; build/distributions/app-3.3.1.tar /&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;ADD&lt;/span&gt;&lt;span class=&quot;s&quot;&gt; config.template.json /app-3.3.1/bin/config.json&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;ADD&lt;/span&gt;&lt;span class=&quot;s&quot;&gt; docker/script/start.sh /usr/local/bin/&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;RUN &lt;/span&gt;ln &lt;span class=&quot;nt&quot;&gt;-s&lt;/span&gt; /usr/local/bin/start.sh /start.sh


&lt;span class=&quot;k&quot;&gt;EXPOSE&lt;/span&gt;&lt;span class=&quot;s&quot;&gt; 8080&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;EXPOSE&lt;/span&gt;&lt;span class=&quot;s&quot;&gt; 7000&lt;/span&gt;


&lt;span class=&quot;k&quot;&gt;CMD&lt;/span&gt;&lt;span class=&quot;s&quot;&gt; [&quot;start.sh&quot;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;도커는 이미지를 만들기 위해 &lt;code class=&quot;highlighter-rouge&quot;&gt;Dockerfile&lt;/code&gt;이라는 파일에 자체 DSLDomain-specific language언어를 이용하여 이미지 생성 과정을 적습니다. 추후에 문법에 대해 자세히 다루겠지만 위 샘플을 보면 그렇게 복잡하지 않다는 걸 알 수 있습니다. 이것은 굉장히 간단하지만 유용한 아이디어인데, 서버에 어떤 프로그램을 설치하려고 이것저것 의존성 패키지를 설치하고 설정파일을 만들었던 경험이 있다면 더 이상 그 과정을 블로깅 하거나 메모장에 적지 말고 &lt;code class=&quot;highlighter-rouge&quot;&gt;Dockerfile&lt;/code&gt;로 관리하면 됩니다. 이 파일은 소스와 함께 버전 관리 되고 원한다면 누구나 이미지 생성과정을 보고 수정할 수 있습니다.&lt;/p&gt;

&lt;h2 id=&quot;도커-설치하기&quot;&gt;도커 설치하기&lt;/h2&gt;

&lt;p&gt;블로그 주인의 경우,  AWS Linux 환경에서 개발을 많이 하기 때문에, Linux 환경으로 설치해보겠습니다.
리눅스에 도커를 설치하는 방법은 자동 설치 스크립트를 이용하는 것이 가장 쉽습니다. 다음 명령어를 입력하면 root 권한을 요구하고 잠시 기다리면 설치가 완료됩니다.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;curl -fsSL https://get.docker.com/ | sudo sh 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;sudo-없이-사용하기&quot;&gt;sudo 없이 사용하기**&lt;/h4&gt;

&lt;p&gt;docker는 기본적으로 root권한이 필요합니다. root가 아닌 사용자가 sudo없이 사용하려면 해당 사용자를 &lt;code class=&quot;highlighter-rouge&quot;&gt;docker&lt;/code&gt;그룹에 추가합니다.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;sudo usermod -aG docker $USER # 현재 접속중인 사용자에게 권한주기
sudo usermod -aG docker your-user # your-user 사용자에게 권한주기
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;컨테이너-실행하기&quot;&gt;컨테이너 실행하기&lt;/h4&gt;

&lt;p&gt;도커를 실행하는 명령어는 다음과 같습니다.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker run [OPTIONS] IMAGE[:TAG|@DIGEST] [COMMAND] [ARG...]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;다음은 자주 사용하는 옵션들입니다.&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;옵션&lt;/th&gt;
      &lt;th&gt;설명&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;-d&lt;/td&gt;
      &lt;td&gt;detached mode 흔히 말하는 백그라운드 모드&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;-p&lt;/td&gt;
      &lt;td&gt;호스트와 컨테이너의 포트를 연결 (포워딩)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;-v&lt;/td&gt;
      &lt;td&gt;호스트와 컨테이너의 디렉토리를 연결 (마운트)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;-e&lt;/td&gt;
      &lt;td&gt;컨테이너 내에서 사용할 환경변수 설정&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;–name&lt;/td&gt;
      &lt;td&gt;컨테이너 이름 설정&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;–rm&lt;/td&gt;
      &lt;td&gt;프로세스 종료시 컨테이너 자동 제거&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;-it&lt;/td&gt;
      &lt;td&gt;-i와 -t를 동시에 사용한 것으로 터미널 입력을 위한 옵션&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;–link&lt;/td&gt;
      &lt;td&gt;컨테이너 연결 [컨테이너명:별칭]&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;h2 id=&quot;도커-기본-명령어&quot;&gt;도커 기본 명령어&lt;/h2&gt;

&lt;h4 id=&quot;컨테이너-목록-확인하기-ps&quot;&gt;컨테이너 목록 확인하기 (ps)&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker ps [OPTIONS]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;컨테이너-중지하기-stop&quot;&gt;컨테이너 중지하기 (stop)&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker stop [OPTIONS] CONTAINER [CONTAINER...]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;blockquote&gt;
  &lt;p&gt;도커 ID의 전체 길이는 64자리 입니다. 하지만 명령어의 인자로 전달할 때는 전부 입력하지 않아도 됩니다. 예를 들어 ID가 abcdefgh…라면 abcd만 입력해도 됩니다. 앞부분이 겹치지 않는다면 1-2자만 입력해도 됩니다.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h4 id=&quot;컨테이너-제거하기-rm&quot;&gt;컨테이너 제거하기 (rm)&lt;/h4&gt;

&lt;p&gt;종료된 컨테이너를 완전히 제거하는 명령어는 다음과 같습니다.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker rm [OPTIONS] CONTAINER [CONTAINER...]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;blockquote&gt;
  &lt;p&gt;중지된 컨테이너 ID를 가져와서 한번에 삭제
docker rm -v $(docker ps -a -q -f status=exited)&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h4 id=&quot;이미지-목록-확인하기-images&quot;&gt;이미지 목록 확인하기 (images)&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker images [OPTIONS] [REPOSITORY[:TAG]]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;이미지-다운로드하기-pull&quot;&gt;이미지 다운로드하기 (pull)&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker pull [OPTIONS] NAME[:TAG|@DIGEST]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;이미지-삭제하기-rmi&quot;&gt;이미지 삭제하기 (rmi)&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker rmi [OPTIONS] IMAGE [IMAGE...]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;images 명령어를 통해 얻은 이미지 목록에서 이미지 ID를 입력하면 삭제가 됩니다. 단, 컨테이너가 실행중인 이미지는 삭제되지 않습니다. 컨테이너는 이미지들의 레이어를 기반으로 실행중이므로 당연히 삭제할 수 없습니다.&lt;/p&gt;

&lt;h4 id=&quot;컨테이너-로그-보기-logs&quot;&gt;컨테이너 로그 보기 (logs)&lt;/h4&gt;

&lt;p&gt;컨테이너가 정상적으로 동작하는지 확인하는 좋은 방법은 로그를 확인하는 것 입니다.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker logs [OPTIONS] CONTAINER
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;컨테이너-명령어-실행하기-exec&quot;&gt;컨테이너 명령어 실행하기 (exec)&lt;/h4&gt;

&lt;p&gt;실행중인 컨테이너에 들어가거나 컨테이너의 파일을 실행하고 싶을 때가 있습니다.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker exec [OPTIONS] CONTAINER COMMAND [ARG...]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;run은 새로 컨테이너를 만들어서 실행하고 exec은 실행중인 컨테이너에 명령어로 실행합니다.&lt;/p&gt;

&lt;h2 id=&quot;도커-이미지-만들기&quot;&gt;도커 이미지 만들기&lt;/h2&gt;

&lt;p&gt;도커는 이미지를 만들기 위해 &lt;strong&gt;컨테이너의 상태를 그대로 이미지로 저장&lt;/strong&gt;하는 방법을 사용합니다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://subicura.com/assets/article_images/2017-02-10-docker-guide-for-beginners-create-image-and-deploy/create-image.png&quot; alt=&quot;create docker image&quot; /&gt;&lt;/p&gt;

&lt;p&gt;어떤 애플리케이션을 이미지로 만든다면 리눅스만 설치된 컨테이너에 애플리케이션을 설치하고 그 상태를 그대로 이미지로 저장합니다. 가상머신의 스냅샷과 비슷한 방식입니다. 이때 핵심 명령어는 파일을 복사하는 &lt;code class=&quot;highlighter-rouge&quot;&gt;COPY&lt;/code&gt;와 명령어를 실행하는 &lt;code class=&quot;highlighter-rouge&quot;&gt;RUN&lt;/code&gt;입니다. DockerFile을 만드는 방법은 실제 쉘 스크립트의 내용을 그대로 옮기면 됩니다.&lt;/p&gt;

&lt;h2 id=&quot;docker-컨테이너container-커밋commit하기&quot;&gt;Docker 컨테이너(Container) 커밋(Commit)하기&lt;/h2&gt;

&lt;p&gt;현재까지 작업해 놓은 ubuntu 컨테이너를 그대로 저장해보자.&lt;/p&gt;

&lt;p&gt;docker commit CONTAINER IMAGE_NAME` 명령 이용.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  $ docker commit A A_image
  sha256:85b8eb5a23e6c850c4f4d298119275ec5a85bc43c78414372aee2859e9ad9e54
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;container 이름이 A 이고, 이를 이미지로 저장할 때 &lt;code class=&quot;highlighter-rouge&quot;&gt;docker images&lt;/code&gt;로 확인하면 다음과 같다.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  $ docker images
  REPOSITORY        TAG                 IMAGE ID            CREATED              SIZE
  A_image           latest              85b8eb5a23e6        About a minute ago   582MB
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;docker-이미지image-push하기&quot;&gt;Docker 이미지(Image) Push하기&lt;/h2&gt;

&lt;p&gt;Docker cloud 에 로그인 하기&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  $ docker login
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;docker user id 변수 지정해놓기&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  $ export DOCKER_ID_USER=&quot;nicewoong&quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Docker Image 에 태그 달기&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  $ docker tag ubuntu-cpp-driver $DOCKER_ID_USER/ubuntu-cpp-driver
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Tag 가 적용되어 있는 Image 를 Docker Cloud 에 Push&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  $ docker push $DOCKER_ID_USER/ubuntu-cpp-driver
  The push refers to a repository [docker.io/nicewoong/ubuntu-cpp-driver]
  441a4ecb7164: Pushing [==================================================&amp;gt;]  472.2MB
  7f7a065d245a: Mounted from nicewoong/ubuntu-bluecoat 
  f96e6b25195f: Mounted from nicewoong/ubuntu-bluecoat 
  c56153825175: Mounted from nicewoong/ubuntu-bluecoat 
  ae620432889d: Mounted from nicewoong/ubuntu-bluecoat 
  a2022691bf95: Mounted from nicewoong/ubuntu-bluecoat 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;dockerfile-1&quot;&gt;DockerFile&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# 1. ubuntu 설치 (패키지 업데이트 + 만든사람 표시)
FROM       ubuntu:16.04
MAINTAINER dwhylee@gmail.com
RUN        apt-get -y update

# 2. A 설치
RUN apt-get -y install A

# 3. 소스 복사
COPY . /usr/src/app

# 4. 패키지 설치 (실행 디렉토리 설정)
WORKDIR /usr/src/app
RUN     bundle install

# 5. 실행 
CMD  ~
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;dockerfile-기본-명령어&quot;&gt;DockerFile 기본 명령어&lt;/h4&gt;

&lt;h5 id=&quot;1-from&quot;&gt;1. FROM&lt;/h5&gt;

&lt;p&gt;​    베이스 이미지를 지정합니다. 반드시 지정해야 하며 어떤 이미지도 베이스 이미지가 될 수 있습니다. tag는 
​    될 수 있으면 기본값보다 구체적인 버전(16.04등)을 지정하는 것이 좋습니다. 이미 만들어진 다양한 베이스 
   이미지는 &lt;a href=&quot;https://hub.docker.com/explore/&quot;&gt;Docker hub&lt;/a&gt;에서 확인할 수 있습니다.&lt;/p&gt;

&lt;h5 id=&quot;2-maintainer&quot;&gt;2. MAINTAINER&lt;/h5&gt;

&lt;p&gt;​    Dockerfile을 관리하는 사람의 이름 또는 이메일 정보를 적습니다. 빌드에 딱히 영향을 주지는 않습니다.&lt;/p&gt;

&lt;h5 id=&quot;3-copy&quot;&gt;3. COPY&lt;/h5&gt;

&lt;p&gt;​    파일이나 디렉토리를 이미지로 복사합니다. 일반적으로 소스를 복사하는 데 사용합니다. 
​    &lt;code class=&quot;highlighter-rouge&quot;&gt;target&lt;/code&gt;디렉토리가 없다면 자동으로 생성합니다.&lt;/p&gt;

&lt;h5 id=&quot;4-add&quot;&gt;4. ADD&lt;/h5&gt;

&lt;p&gt;​    &lt;code class=&quot;highlighter-rouge&quot;&gt;COPY&lt;/code&gt;명령어와 매우 유사하나 몇가지 추가 기능이 있습니다. &lt;code class=&quot;highlighter-rouge&quot;&gt;src&lt;/code&gt;에 파일 대신 URL을 입력할 수 있고 
​    &lt;code class=&quot;highlighter-rouge&quot;&gt;src&lt;/code&gt;에 압축 파일을 입력하는 경우 자동으로 압축을 해제하면서 복사됩니다.&lt;/p&gt;

&lt;h5 id=&quot;5-run&quot;&gt;5. RUN&lt;/h5&gt;

&lt;p&gt;​    가장 많이 사용하는 구문입니다. 명령어를 그대로 실행합니다. 내부적으로 &lt;code class=&quot;highlighter-rouge&quot;&gt;/bin/sh -c&lt;/code&gt; 뒤에 명령어를 
​    실행하는 방식입니다.&lt;/p&gt;

&lt;h5 id=&quot;6-cmd&quot;&gt;6. CMD&lt;/h5&gt;

&lt;p&gt;​    도커 컨테이너가 실행되었을 때 실행되는 명령어를 정의합니다. 빌드할 때는 실행되지 않으며 여러 개의 &lt;code class=&quot;highlighter-rouge&quot;&gt;CMD&lt;/code&gt;가
​    존재할 경우 가장 마지막 &lt;code class=&quot;highlighter-rouge&quot;&gt;CMD&lt;/code&gt;만 실행됩니다. 한꺼번에 여러 개의 프로그램을 실행하고 싶은 경우에는 &lt;code class=&quot;highlighter-rouge&quot;&gt;run.sh&lt;/code&gt;
​    파일을 작성하여 데몬으로 실행하거나 &lt;a href=&quot;http://supervisord.org/&quot;&gt;supervisord&lt;/a&gt;, &lt;a href=&quot;https://github.com/ddollar/forego&quot;&gt;forego&lt;/a&gt;와 같은 여러 개의 프로그램을 실행하는 프로그램을 
​    사용합니다.&lt;/p&gt;

&lt;h5 id=&quot;7-workdir&quot;&gt;7. WORKDIR&lt;/h5&gt;

&lt;p&gt;​    RUN, CMD, ADD, COPY등이 이루어질 기본 디렉토리를 설정합니다. 각 명령어의 현재 디렉토리는 한 줄마다 
​    초기화되기 때문에 &lt;code class=&quot;highlighter-rouge&quot;&gt;RUN cd /path&lt;/code&gt;를 하더라도 다음 명령어에선 위치가 초기화 됩니다. 같은 디렉토리에서 
​    계속 작업하기 위해서 &lt;code class=&quot;highlighter-rouge&quot;&gt;WORKDIR&lt;/code&gt;을 사용합니다&lt;/p&gt;

&lt;h5 id=&quot;8-expose&quot;&gt;8. EXPOSE&lt;/h5&gt;

&lt;p&gt;​    도커 컨테이너가 실행되었을 때 요청을 기다리고 있는(Listen) 포트를 지정합니다. 여러개의 포트를 지정할 수 
​    있습니다.&lt;/p&gt;

&lt;h5 id=&quot;9-volume&quot;&gt;9. VOLUME&lt;/h5&gt;

&lt;p&gt;​    컨테이너 외부에 파일시스템을 마운트 할 때 사용합니다. 반드시 지정하지 않아도 마운트 할 수 있지만, 
​    기본적으로 지정하는 것이 좋습니다.&lt;/p&gt;

&lt;h5 id=&quot;10-env&quot;&gt;10. ENV&lt;/h5&gt;

&lt;p&gt;​    컨테이너에서 사용할 환경변수를 지정합니다. 컨테이너를 실행할 때 &lt;code class=&quot;highlighter-rouge&quot;&gt;-e&lt;/code&gt;옵션을 사용하면 기존 값을 오버라이딩
​    하게 됩니다.&lt;/p&gt;

&lt;p&gt;여기까지 Dockerfile에서 가장 많이 사용하는 명령어에 대해 알아보았습니다. 모든 명령어가 궁금하신 분은 &lt;a href=&quot;https://docs.docker.com/engine/reference/builder/&quot;&gt;공식문서&lt;/a&gt;를 참고하시면 됩니다.&lt;/p&gt;

&lt;h4 id=&quot;docker-빌드하기&quot;&gt;Docker 빌드하기&lt;/h4&gt;

&lt;pre&gt;&lt;code class=&quot;language-txt&quot;&gt;docker build [OPTIONS] PATH | URL | -
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;생성할 이미지 이름을 지정하기 위한 &lt;code class=&quot;highlighter-rouge&quot;&gt;-t(--tag)&lt;/code&gt; 옵션&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;INPUT : 
Docker build -t app .

Output :
Sending build context to Docker daemon 22.02 kB
Step 1/10 : FROM ubuntu:16.04
 ---&amp;gt; f49eec89601e
Step 2/10 : MAINTAINER subicura@subicura.com
 ---&amp;gt; Running in 06f20ac1017d
 ---&amp;gt; fc41cd8ac52d
Removing intermediate container 06f20ac1017d
Step 3/10 : RUN apt-get -y update
 ---&amp;gt; Running in c35738e75a51
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial/main Sources [1103 kB]
Get:5 http://archive.ubuntu.com/ubuntu xenial/restricted Sources [5179 B]

... 생략 ...

Step 9/10 : EXPOSE 4567
 ---&amp;gt; Running in 9c514a1f0c8e
 ---&amp;gt; c5ce4376282e
Removing intermediate container 9c514a1f0c8e
Step 10/10 : CMD bundle exec ruby app.rb -o 0.0.0.0
 ---&amp;gt; Running in 1f7a9ba8d63c
 ---&amp;gt; 54d239c00f11
Removing intermediate container 1f7a9ba8d63c
Successfully built 54d239c00f11
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;빌드 명령어를 실행하면 Dockerfile에 정의한 내용이 한 줄 한 줄 실행되는 걸 볼 수 있습니다.  실제로 명령어를 실행하기 때문에 빌드 시간이 꽤 걸리는 걸 알 수 있습니다.  도커 빌드는 임시 컨테이너 생성 &amp;gt; 명령어 수행 &amp;gt; 이미지로 저장 &amp;gt; 임시 컨테이너 삭제 &amp;gt; 새로 만든 이미지 기반 임시 컨테이너 생성 &amp;gt; 명령어 수행 &amp;gt; 이미지로 저장 &amp;gt; 임시 컨테이너 삭제 &amp;gt; … 의 과정을 계속해서 반복합니다. 명령어를 실행할 때마다 이미지 레이어를 저장하고 다시 빌드할 때 Dockerfile이 변경되지 않았다면 기존에 저장된 이미지를 그대로 &lt;strong&gt;“캐시”&lt;/strong&gt;처럼 사용합니다. 이러한 레이어 개념을 잘 이해하고 있어야 최적화된 이미지를 생성할 수 있습니다.&lt;/p&gt;

&lt;p&gt;최종적으로 &lt;code class=&quot;highlighter-rouge&quot;&gt;Successfully built xxxxxxxx&lt;/code&gt; 메시지가 보이면 정상적으로 이미지를 생성한 것입니다. 이후 &lt;code class=&quot;highlighter-rouge&quot;&gt;docker images&lt;/code&gt;를 통해 확인하면 이미지가 생성된 것을 확인할 수 있고, 잘 동작하는지 컨테이너를 실행하면 됩니다.&lt;/p&gt;

&lt;h4 id=&quot;도커-이미지-리팩토링-및-명령어-최적화&quot;&gt;도커 이미지 리팩토링 및 명령어 최적화&lt;/h4&gt;

&lt;p&gt;이미지를 빌드하는 과정에서 각 단계를 이미지 레이어로 저장하고 다음 빌드에서 캐시로 사용합니다. 도커는 빌드할 때 Dockerfile의 명령어가 수정되었거나 추가하는 파일이 변경 되었을 때 캐시가 깨지고 그 이후 작업은 새로 이미지를 만들게 됩니다. 또한 이미지를 빌드할 때 불필요한 로그는 무시하는게 좋고 패키지 설치시 문서 파일도 생성할 필요가 없습니다. (&lt;code class=&quot;highlighter-rouge&quot;&gt;-qq&lt;/code&gt; 옵션 또는 &lt;code class=&quot;highlighter-rouge&quot;&gt;quite&lt;/code&gt; 옵션 , &lt;code class=&quot;highlighter-rouge&quot;&gt;--no-doc&lt;/code&gt;과 &lt;code class=&quot;highlighter-rouge&quot;&gt;--no-ri&lt;/code&gt; 옵션 등)&lt;/p&gt;

&lt;h2 id=&quot;컨테이너-업데이트&quot;&gt;컨테이너 업데이트&lt;/h2&gt;

&lt;p&gt;&lt;img src=&quot;https://subicura.com/assets/article_images/2017-01-19-docker-guide-for-beginners-2/container-update.png&quot; alt=&quot;container&quot; /&gt;&lt;/p&gt;

&lt;p&gt;도커에서 컨테이너를 업데이트 하려면 새 버전의 이미지를 다운(pull)받고 기존 컨테이너를 삭제(stop, rm) 한 후 새 이미지를 기반으로 새 컨테이너를 실행(run)하면 됩니다. 컨테이너 삭제시 유지해야하는 데이터는 반드시 컨테이너 내부가 아닌 외부 스토리지에 저장해야 합니다. 가장 좋은 방법은 &lt;a href=&quot;http://docs.aws.amazon.com/ko_kr/AmazonS3/latest/dev/Welcome.html&quot;&gt;AWS S3&lt;/a&gt;같은 클라우드 서비스를 이용하는 것이고 그렇지 않으면 데이터 볼륨(Data volumes)을 컨테이너에 추가해서 사용해야 합니다. 데이터 볼륨을 사용하면 해당 디렉토리는 컨테이너와 별도로 저장되고 컨테이너를 삭제해도 데이터가 지워지지 않습니다.&lt;/p&gt;

&lt;h2 id=&quot;docker-compose&quot;&gt;Docker Compose&lt;/h2&gt;

&lt;p&gt;도커는 복잡한 설정을 쉽게 관리하기 위해 &lt;a href=&quot;https://en.wikipedia.org/wiki/YAML&quot;&gt;YAML&lt;/a&gt;방식의 설정파일을 이용한 &lt;a href=&quot;https://docs.docker.com/compose/&quot;&gt;Docker Compose&lt;/a&gt;라는 툴을 제공합니다.  Dokcer Compose에서는 컨테이너 실행에 사용되는 옵션과 컨테이너 간 의존성을 모두 docker-compose.yml파일에 적어두고, 컴포즈용 명령어를 사용하여 컨테이너들을 실행, 관리합니다.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;docker build &lt;em&gt;vs.&lt;/em&gt; docker-compose build: 도커 이미지 만들기(=&lt;strong&gt;클래스 선언을 애플리케이션에 로드&lt;/strong&gt;)&lt;/li&gt;
  &lt;li&gt;docker run의 옵션들 &lt;em&gt;vs.&lt;/em&gt; docker-compose.yml: 이미지에 붙이는 장식들(=&lt;strong&gt;인스턴스의 변수들&lt;/strong&gt;)&lt;/li&gt;
  &lt;li&gt;docker run &lt;em&gt;vs.&lt;/em&gt; docker-compose up: 장식 붙은 이미지를 실제로 실행(=&lt;strong&gt;인스턴스 생성&lt;/strong&gt;)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;클래스와 인스턴스의 예를 들면, 어떤 인스턴스를 자주 생성해야 해서 필요한 변수를 딕셔너리로 저장해두는 것과 비슷하겠습니다. 이를 그림으로 나타내면 다음과 같습니다.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;http://raccoonyy.github.io/content/images/2017/03/docker-compose-yml.png&quot; alt=&quot;img&quot; /&gt;&lt;/p&gt;

&lt;h4 id=&quot;설치하기&quot;&gt;설치하기&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;curl -L &quot;https://github.com/docker/compose/releases/download/1.9.0/docker-compose-$(uname -s)-$(uname -m)&quot; -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
# test
docker-compose version
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;docker-composeyml&quot;&gt;docker-compose.yml&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;version: '3.1'

service:
	servicename:
		image: # optional
		command: # optional
		environment: # optional
		volumes # optional
	servicename2: # if have second service...
    volumes: # optional

    network: # optional
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h5 id=&quot;1-version&quot;&gt;1. version&lt;/h5&gt;

&lt;p&gt;​    docker-compose.yml 파일의 제일 윗부분에는 파일 규격 버전을 적습니다.&lt;/p&gt;

&lt;h5 id=&quot;2-services&quot;&gt;2. services&lt;/h5&gt;

&lt;p&gt;​    실행하려는 서비스들을 정의합니다. 서비스란, 앱 컨테이너나 postgres 컨테이너 각각의 묶음입니다.&lt;/p&gt;

&lt;h5 id=&quot;3-image&quot;&gt;3. image&lt;/h5&gt;

&lt;p&gt;​    서비스에서 사용할 도커 이미지를 적습니다.&lt;/p&gt;

&lt;h5 id=&quot;4-volumes&quot;&gt;4. volumes&lt;/h5&gt;

&lt;p&gt;​    docker run으로 앱 컨테이너를 실행할 때 –volume 옵션을 사용하여 데이터베이스의 데이터를 로컬 컴퓨터에 
​    저장했던 부분과 같습니다. 다만 docker-compose.yml에서는  상대 경로를 지정할 수 있다는 점이 다릅니다.&lt;/p&gt;

&lt;h5 id=&quot;5-environment&quot;&gt;5. environment&lt;/h5&gt;

&lt;p&gt;​    docker run에서 -e 옵션에 적었던 내용들입니다.&lt;/p&gt;

&lt;h5 id=&quot;6-build&quot;&gt;6. build&lt;/h5&gt;

&lt;p&gt;​    앱 서비스는 도커 이미지를 빌드할 일이 잦기 때문에, 특정 이미지 대신 build 옵션을 추가합니다.&lt;/p&gt;

&lt;h5 id=&quot;7-port&quot;&gt;7. port&lt;/h5&gt;

&lt;p&gt;​    도커 컴포즈로 서비스를 실행하면 기본적으로 가상의 네트워크가 하나 만들어지고, 네트워크 외부의 접근이 
​    막힙니다. 따라서 외부에서 접근할 수 있는 포트를 지정해주어야 합니다.&lt;/p&gt;

&lt;h5 id=&quot;8-depends_on&quot;&gt;8. depends_on&lt;/h5&gt;

&lt;p&gt;​    서비스가 하나 이상일 때, 실행 의존성을 지정할 수 있습니다.&lt;/p&gt;

&lt;h5 id=&quot;9-links&quot;&gt;9. links&lt;/h5&gt;

&lt;p&gt;​    docker run에서 –link 옵션에서 지정했던 내용입니다&lt;/p&gt;

&lt;h5 id=&quot;10-command&quot;&gt;10. command&lt;/h5&gt;

&lt;p&gt;docker run으로 앱 컨테이너를 실행하는 command 입니다.&lt;/p&gt;

&lt;p&gt;이 외에 자세한 사항은 &lt;a href=&quot;https://docs.docker.com/compose/compose-file/&quot;&gt;docker-compose&lt;/a&gt;를 참조하시면 됩니다.&lt;/p&gt;

&lt;h4 id=&quot;docker-composeyml-관리법&quot;&gt;docker-compose.yml 관리법&lt;/h4&gt;

&lt;p&gt;&lt;strong&gt;docker-compose.yml&lt;/strong&gt; : 이미지만 설정되어있는 베이스 파일&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;version: '3.1'

services:

  drupal:
    image: drupal:latest

  postgres:
    image: postgres:9.6
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;docker-compose.override.yml&lt;/strong&gt; : 개발시 사용되는 config&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;version: '3.1'

services:

  drupal:
    build: .
    ports:
      - &quot;8080:80&quot;
    volumes:
      - drupal-modules:/var/www/html/modules
      - drupal-profiles:/var/www/html/profiles
      - drupal-sites:/var/www/html/sites
      - ./themes:/var/www/html/themes

  postgres:
    environment:
      - POSTGRES_PASSWORD_FILE=/run/secrets/psql-pw
    secrets:
      - psql-pw
    volumes:
      - drupal-data:/var/lib/postgresql/data

  volumes:
    drupal-data:
    drupal-modules:
    drupal-profiles:
    drupal-sites:
    drupal-themes:

  secrets:
    psql-pw:
      file: psql-fake-password.txt
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;docker-compose.prod.yml&lt;/strong&gt; : 실제 배포시 사용되는 config&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;version: '3.1'

services:

  drupal:
    ports:
      - &quot;80:80&quot;
    volumes:
      - drupal-modules:/var/www/html/modules
      - drupal-profiles:/var/www/html/profiles
      - drupal-sites:/var/www/html/sites
      - drupal-themes:/var/www/html/themes

  postgres:
    environment:
      - POSTGRES_PASSWORD_FILE=/run/secrets/psql-pw
    secrets:
      - psql-pw
    volumes:
      - drupal-data:/var/lib/postgresql/data

  volumes:
    drupal-data:
    drupal-modules:
    drupal-profiles:
    drupal-sites:
    drupal-themes:

secrets:
  psql-pw:
    external: true
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;docker-compose.test.yml&lt;/strong&gt; : CI에서 사용되는 config&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;version: '3.1'

services:

  drupal:
    image: drupal
    build: .
    ports:
      - &quot;80:80&quot;

  postgres:
    environment:
      - POSTGRES_PASSWORD_FILE=/run/secrets/psql-pw
    secrets:
      - psql-pw
    volumes:
      - ./sample-data:/var/lib/postgresql/data
secrets:
  psql-pw:
    file: psql-fake-password.txt
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;이미지-저장소&quot;&gt;이미지 저장소&lt;/h2&gt;

&lt;p&gt;&lt;img src=&quot;https://subicura.com/assets/article_images/2017-02-10-docker-guide-for-beginners-create-image-and-deploy/docker-registry.png&quot; alt=&quot;Docker Registry&quot; /&gt;&lt;/p&gt;

&lt;p&gt;도커는 빌드한 이미지를 서버에 배포하기 위해 직접 파일을 복사하는 방법 대신 &lt;a href=&quot;https://docs.docker.com/registry/&quot;&gt;도커 레지스트리&lt;/a&gt;(Docker Registry)라는 이미지 저장소를 사용합니다. 도커 명령어를 이용하여 이미지를 레지스트리에 푸시(push)하고 다른 서버에서 풀(pull)받아 사용하는 구조입니다. (git과 유사)&lt;/p&gt;

&lt;h2 id=&quot;배포하기&quot;&gt;배포하기&lt;/h2&gt;

&lt;h4 id=&quot;컨테이너-배포-방식으로&quot;&gt;컨테이너 배포 방식으로&lt;/h4&gt;

&lt;p&gt;컨테이너를 배포하는 방식은 기존에 애플리케이션을 배포하는 방식과 큰 차이가 있습니다. 기존에 애플리케이션을 배포하는 방식은 사용하는 언어, 프레임워크, 웹(or WAS)서버, 리눅스 배포판, 개발자의 취향에 따라 각각 다른 방식을 사용했습니다. 새로운 서버를 셋팅하고 한 번에 배포를 성공한다는 건 굉장히 힘든 일이었고 의존성 라이브러리가 제대로 설치되었는지 검증하기도 매우 어려웠습니다. ftp, rsync, ant, gradle, capistrano, fabric, chef, puppet, ansible등 다양한 배포툴이 저마다의 장점을 가지고 등장하였고 배포하는 방식을 하나로 정의한다는 건 거의 불가능했습니다. 하지만, 컨테이너를 사용하면 어떤 언어, 어떤 프레임워크를 쓰든 상관없이 배포 방식이 동일해지고 과정 또한 굉장히 단순해집니다. 그냥 &lt;strong&gt;이미지를 다운받고 컨테이너를 실행하면 끝입니다.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://subicura.com/assets/article_images/2017-02-10-docker-guide-for-beginners-create-image-and-deploy/docker-update.png&quot; alt=&quot;Update&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;참고-자료&quot;&gt;참고 자료&lt;/h2&gt;

&lt;p&gt;https://cloud.google.com/containers/&lt;/p&gt;

&lt;p&gt;https://www.sauru.so/blog/getting-started-with-docker/&lt;/p&gt;

&lt;p&gt;https://subicura.com/2017/01/19/docker-guide-for-beginners-1.html&lt;/p&gt;

&lt;p&gt;https://subicura.com/2017/01/19/docker-guide-for-beginners-2.html&lt;/p&gt;

&lt;p&gt;https://subicura.com/2017/01/19/docker-guide-for-beginners-3.html&lt;/p&gt;

&lt;p&gt;https://gist.github.com/kkweon/2359da953f9b127b19c4146060797631&lt;/p&gt;

&lt;p&gt;https://docs.docker.com/compose/compose-file/&lt;/p&gt;
</description>
        <pubDate>Thu, 14 Mar 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/docker/image/usage/2019/03/14/Docker%EC%A0%95%EB%A6%AC/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/docker/image/usage/2019/03/14/Docker%EC%A0%95%EB%A6%AC/</guid>
        
        <category>docker</category>
        
        <category>image</category>
        
        <category>usage</category>
        
        
        <category>docker</category>
        
        <category>image</category>
        
        <category>usage</category>
        
      </item>
    
      <item>
        <title>[딥러닝] 국가건강검진 데이터를 활용한 당뇨병/고혈압 예측</title>
        <description>
&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54178151-28399d80-44d8-11e9-8438-1f70586544f0.png&quot; alt=&quot;1552370311347&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54178037-b3666380-44d7-11e9-9733-31a847401e79.png&quot; alt=&quot;1552370447146&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54178416-14426b80-44d9-11e9-82b2-3000627549c0.png&quot; alt=&quot;1552370571695&quot; /&gt;&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;import tensorflow as tf
import numpy as np
import os
import time
import matplotlib.pyplot as plt

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
tf.set_random_seed(777)   # for reproducibility
nb_classes = 4

def read(input):
    data = np.loadtxt(input, delimiter=',', dtype=np.float32)
    data = (data - data.min(axis=0)) / (data.max(axis=0) - data.min(axis=0))
    x_data = data[:, 0:-1]
    y_data = data[:, [-1]]
    return x_data, y_data

def layer():
    X = tf.placeholder(tf.float32, [None, 5])
    Y = tf.placeholder(tf.int32, [None, 1])

    W1 = tf.get_variable(&quot;weight1&quot;, shape=[5, 10], initializer=tf.contrib.layers.xavier_initializer())
    b1 = tf.Variable(tf.random_normal([10]), name='bias1')
    layer1 =  tf.nn.tanh(tf.matmul(X, W1) + b1)

    W2 = tf.get_variable(&quot;weight2&quot;, shape=[10, 10], initializer=tf.contrib.layers.xavier_initializer())
    b2 = tf.Variable(tf.random_normal([10]), name='bias2')
    layer2 = tf.nn.tanh(tf.matmul(layer1, W2) + b2)

    W3 = tf.get_variable(&quot;weight3&quot;, shape=[10, 10], initializer=tf.contrib.layers.xavier_initializer())
    b3 = tf.Variable(tf.random_normal([10]), name='bias3')
    layer3 = tf.nn.tanh(tf.matmul(layer2,W3) + b3)

    W4 = tf.get_variable(&quot;weight4&quot;, shape=[10, 10], initializer=tf.contrib.layers.xavier_initializer())
    b4 = tf.Variable(tf.random_normal([10]), name='bias4')
    layer4 = tf.nn.tanh(tf.matmul(layer3,W4) + b4)

    W5 = tf.get_variable(&quot;weight5&quot;, shape=[10, nb_classes], initializer=tf.contrib.layers.xavier_initializer())
    b5 = tf.Variable(tf.random_normal([nb_classes]), name='bias5')
    hypothesis = tf.nn.softmax(tf.matmul(layer4,W5) + b5)

    return X, Y, hypothesis

def cost(Y_one_hot, hypothesis):
    cost_op = tf.reduce_mean(-tf.reduce_sum(Y_one_hot * tf.log(hypothesis), axis =1))
    return cost_op

def trainning(loss_op, learning_rate):
    optimizer = tf.train.AdamOptimizer(learning_rate)
    train_op = optimizer.minimize(loss_op)
    return train_op

def evaluation(Y_one_hot, hypothesis):
    prediction = tf.argmax(hypothesis, 1)
    corrected_prediction = tf.equal(prediction, tf.argmax(Y_one_hot, 1))
    accuracy = tf.reduce_mean(tf.cast(corrected_prediction, tf.float32))
    return accuracy

if __name__ == &quot;__main__&quot;:
    start_time = time.time()
    input, output = read('learning_data.csv')
    testInput, testOutput = read('test_data.csv')

    X, Y, logits = layer()
    Y_one_hot = tf.reshape(tf.one_hot(Y, nb_classes), [-1, nb_classes])  # reshape 해준다
    cost_op = cost(Y_one_hot,logits)
    train = trainning(cost_op, 0.3)
    acc = evaluation(Y_one_hot, logits)
    lcost, lcost2, lacc= [], [], []
    xindex= []
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        for step in range(1000):
            cost, acc_learn, _ = sess.run([cost_op, acc, train] , feed_dict={X: input, Y: output})
            cost2 = sess.run(cost_op, feed_dict = {X : testInput, Y:testOutput})
            lcost.append(cost)
            lacc.append(acc_learn)
            print(&quot;Step: {:5d} \tcost: {:.5f} \tAcc_learn: {:.5f}%&quot;.format(step + 1, cost, acc_learn * 100))
            xindex.append(step)
        acc_test = sess.run(acc, feed_dict={X: testInput, Y: testOutput })
        print(&quot;Acc_test: {:.5f}%&quot;.format(acc_test * 100))
        print(&quot;--- %s seconds ---&quot; % (time.time() - start_time))
    plt.plot(xindex,lacc, 'red', label = 'acc_learn')
    plt.ylim(0,1)
    plt.ylabel('acc_learn')
    plt.show()

    plt.plot(xindex,lcost,'blue', label = 'cost')
    plt.ylim(0, 3)
    plt.ylabel('cost')
    plt.show()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

</description>
        <pubDate>Tue, 12 Mar 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/deep%20learning/python/tensorflow/2019/03/12/%EB%94%A5%EB%9F%AC%EB%8B%9D-%EC%8B%A4%EC%8A%B5/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/deep%20learning/python/tensorflow/2019/03/12/%EB%94%A5%EB%9F%AC%EB%8B%9D-%EC%8B%A4%EC%8A%B5/</guid>
        
        <category>deep learning</category>
        
        <category>python</category>
        
        <category>tensorflow</category>
        
        
        <category>deep learning</category>
        
        <category>python</category>
        
        <category>tensorflow</category>
        
      </item>
    
      <item>
        <title>[논문리뷰] Parallel Multiscale Autoregressive Density Estimation</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#parallel-multiscale-autoregressive-density-estimation&quot; id=&quot;markdown-toc-parallel-multiscale-autoregressive-density-estimation&quot;&gt;Parallel Multiscale Autoregressive Density Estimation&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;parallel-multiscale-autoregressive-density-estimation&quot;&gt;Parallel Multiscale Autoregressive Density Estimation&lt;/h1&gt;

&lt;p&gt;2018-11-12, Lab Seminar Paper review&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54176607-e2c6a180-44d2-11e9-852e-a01465372fdb.png&quot; alt=&quot;1552368396547&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54176647-038ef700-44d3-11e9-87ee-6099ca83a57c.png&quot; alt=&quot;1552368473624&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54176722-49e45600-44d3-11e9-81e4-9d3e1f561257.png&quot; alt=&quot;1552368581864&quot; /&gt;&lt;/p&gt;

&lt;p&gt;​						추가적으로 공부해볼만한 분야 (Variational Autoencoder, GAN)&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54176801-929c0f00-44d3-11e9-820a-7c5197694da9.png&quot; alt=&quot;1552368654999&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54176886-d2fb8d00-44d3-11e9-8315-693c9dc88606.png&quot; alt=&quot;1552368808822&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54176932-f0305b80-44d3-11e9-8721-6e13ab33b904.png&quot; alt=&quot;1552368853970&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54176958-063e1c00-44d4-11e9-9f9b-6b5c64eb70e7.png&quot; alt=&quot;1552368891187&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54177014-34bbf700-44d4-11e9-8de1-3aaf9826d349.png&quot; alt=&quot;1552368939156&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54177068-5917d380-44d4-11e9-8019-828adcf8a02c.png&quot; alt=&quot;1552369029229&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54177099-6b920d00-44d4-11e9-96e8-9fd99fdd73e0.png&quot; alt=&quot;1552369059060&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54177115-777dcf00-44d4-11e9-891e-b771e33f4ca4.png&quot; alt=&quot;1552369081397&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54177131-85335480-44d4-11e9-88c6-6d1978a1c510.png&quot; alt=&quot;1552369104266&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54177149-92504380-44d4-11e9-8db7-ba1fe9eb7a52.png&quot; alt=&quot;1552369126479&quot; /&gt;&lt;/p&gt;

</description>
        <pubDate>Tue, 12 Mar 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/machine%20learning/image%20generation%20model/paper%20review/2019/03/12/paper-review1/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/machine%20learning/image%20generation%20model/paper%20review/2019/03/12/paper-review1/</guid>
        
        <category>machine learning</category>
        
        <category>image generation model</category>
        
        <category>paper review</category>
        
        
        <category>machine learning</category>
        
        <category>image generation model</category>
        
        <category>paper review</category>
        
      </item>
    
      <item>
        <title>[JAVA] JVM 메모리 구조와 JAVA Option값 정리</title>
        <description>
&lt;p&gt;bioinfomatic를 하다보면 Java로 프로그램을 돌리는 경우가 종종 있다.&lt;/p&gt;

&lt;p&gt;그때 자주 발생했던 메모리 문제 한번 정리해보았다.&lt;/p&gt;
</description>
        <pubDate>Tue, 12 Mar 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/jvm/java/tip/bioinfomatics/2019/03/12/JVM%EA%B5%AC%EC%A1%B0-%EB%B0%8F-JAVA-%EC%98%B5%EC%85%98/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/jvm/java/tip/bioinfomatics/2019/03/12/JVM%EA%B5%AC%EC%A1%B0-%EB%B0%8F-JAVA-%EC%98%B5%EC%85%98/</guid>
        
        <category>jvm</category>
        
        <category>java</category>
        
        <category>tip</category>
        
        <category>bioinfomatics</category>
        
        
        <category>jvm</category>
        
        <category>java</category>
        
        <category>tip</category>
        
        <category>bioinfomatics</category>
        
      </item>
    
      <item>
        <title>[블로그] github에 이미지 저장 없이 업로드하기.</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#1-github-저장소에-들어갑니다&quot; id=&quot;markdown-toc-1-github-저장소에-들어갑니다&quot;&gt;1. GitHub 저장소에 들어갑니다&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#2-issues-탭에서-new-issue-클릭&quot; id=&quot;markdown-toc-2-issues-탭에서-new-issue-클릭&quot;&gt;2. Issues 탭에서 “New Issue” 클릭&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#3-text-area에-이미지-올리기&quot; id=&quot;markdown-toc-3-text-area에-이미지-올리기&quot;&gt;3. text area에 이미지 올리기&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#4-해당-마크다운-코드를-url에-복붙하면-끝&quot; id=&quot;markdown-toc-4-해당-마크다운-코드를-url에-복붙하면-끝&quot;&gt;4. 해당 마크다운 코드를 URL에 복붙하면 끝&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;단순 블로그용으로 사용하기 좋은 방법인듯&lt;/p&gt;

&lt;h3 id=&quot;1-github-저장소에-들어갑니다&quot;&gt;1. GitHub 저장소에 들어갑니다&lt;/h3&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54103989-c0208400-4411-11e9-84e8-d729ea6cd647.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;2-issues-탭에서-new-issue-클릭&quot;&gt;2. Issues 탭에서 “New Issue” 클릭&lt;/h3&gt;

&lt;h3 id=&quot;3-text-area에-이미지-올리기&quot;&gt;3. text area에 이미지 올리기&lt;/h3&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54104031-da5a6200-4411-11e9-93f3-936a8b6de005.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Ctrl+c / Ctrl+v 하면 마크다운/HTML 코드가 나타남.&lt;/p&gt;

&lt;h3 id=&quot;4-해당-마크다운-코드를-url에-복붙하면-끝&quot;&gt;4. 해당 마크다운 코드를 URL에 복붙하면 끝&lt;/h3&gt;

</description>
        <pubDate>Mon, 11 Mar 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/blog/github/tip/2019/03/11/%EC%9D%B4%EB%AF%B8%EC%A7%80-%EC%97%85%EB%A1%9C%EB%93%9C/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/blog/github/tip/2019/03/11/%EC%9D%B4%EB%AF%B8%EC%A7%80-%EC%97%85%EB%A1%9C%EB%93%9C/</guid>
        
        <category>blog</category>
        
        <category>github</category>
        
        <category>tip</category>
        
        
        <category>blog</category>
        
        <category>github</category>
        
        <category>tip</category>
        
      </item>
    
      <item>
        <title>[Graph] 응용 그래프(Applied Graph Theory)</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#응용그래프-applied-graph-theory---week-1&quot; id=&quot;markdown-toc-응용그래프-applied-graph-theory---week-1&quot;&gt;응용그래프 (Applied Graph Theory) - Week 1&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#강의-개요와-목표&quot; id=&quot;markdown-toc-강의-개요와-목표&quot;&gt;강의 개요와 목표&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#graph-algorithm-tool-&quot; id=&quot;markdown-toc-graph-algorithm-tool-&quot;&gt;Graph Algorithm Tool (★)&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#leda&quot; id=&quot;markdown-toc-leda&quot;&gt;LEDA&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#boost&quot; id=&quot;markdown-toc-boost&quot;&gt;Boost&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#story-brook-algorithm-repository&quot; id=&quot;markdown-toc-story-brook-algorithm-repository&quot;&gt;Story Brook algorithm repository&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#graph-visualization-tool-&quot; id=&quot;markdown-toc-graph-visualization-tool-&quot;&gt;Graph Visualization Tool (★)&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#graphviz&quot; id=&quot;markdown-toc-graphviz&quot;&gt;Graphviz&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#networkx-python&quot; id=&quot;markdown-toc-networkx-python&quot;&gt;Networkx (python)&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#graphtea-software&quot; id=&quot;markdown-toc-graphtea-software&quot;&gt;GraphTea (software)&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#실습&quot; id=&quot;markdown-toc-실습&quot;&gt;실습&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#과제&quot; id=&quot;markdown-toc-과제&quot;&gt;과제&lt;/a&gt;        &lt;ul&gt;
          &lt;li&gt;&lt;a href=&quot;#기하-그래프의-최단거리&quot; id=&quot;markdown-toc-기하-그래프의-최단거리&quot;&gt;기하 그래프의 최단거리&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#코드&quot; id=&quot;markdown-toc-코드&quot;&gt;코드&lt;/a&gt;&lt;/li&gt;
          &lt;li&gt;&lt;a href=&quot;#결과&quot; id=&quot;markdown-toc-결과&quot;&gt;결과&lt;/a&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;응용그래프-applied-graph-theory---week-1&quot;&gt;응용그래프 (Applied Graph Theory) - Week 1&lt;/h1&gt;

&lt;h3 id=&quot;강의-개요와-목표&quot;&gt;강의 개요와 목표&lt;/h3&gt;

&lt;p&gt;그래프 이론은 컴퓨터 과학의 가장 기본적이며 중요한 도구이다. 그래프 이론은 문제의 모형만들기와 그 해결에 사용되고 있다. 강의의 목표는 2가지로 하나는 그래프 이론 자체에 대한 이해로서 수학적인 접근을 해야 한다. 그 다음에는 실제 현실적인 문제를 그래프로 모형화하고 이것을 해결하는 그래프 알고리즘을 개발하는 것이다.&lt;/p&gt;

&lt;p&gt;이번 강의의 목적은 컴퓨터 과학(공학)에서 흔히 볼 수 있는 여러 계산문제를 그래프에 기반하여 모델링하고,
이를 바탕으로 미리 개발된 그래프 알고리즘을 활용하여 해결하는 것이다&lt;/p&gt;

&lt;p&gt;수강생들은 자신이 처음부터 모든 것을 설계, 구현하는 식으로 코드를 작성하려고 생각하지 말고 알고리즘의 
기본적인 원리를 습득한 뒤,  이미 개발된 알고리즘를 활용하여 보다 높은 차원의 문제를 해결하면 실제 현장에서 효율적이다.&lt;/p&gt;

&lt;h3 id=&quot;graph-algorithm-tool-&quot;&gt;Graph Algorithm Tool (★)&lt;/h3&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;h5 id=&quot;leda&quot;&gt;&lt;a href=&quot;http://www.algorithmic-solutions.info/leda_guide/Index.html&quot;&gt;LEDA&lt;/a&gt;&lt;/h5&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;h5 id=&quot;boost&quot;&gt;&lt;a href=&quot;https://www.boost.org/&quot;&gt;Boost&lt;/a&gt;&lt;/h5&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;h5 id=&quot;story-brook-algorithm-repository&quot;&gt;&lt;a href=&quot;http://algorist.com/algorist.html&quot;&gt;Story Brook algorithm repository&lt;/a&gt;&lt;/h5&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;h3 id=&quot;graph-visualization-tool-&quot;&gt;Graph Visualization Tool (★)&lt;/h3&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;h5 id=&quot;graphviz&quot;&gt;&lt;a href=&quot;https://www.graphviz.org/&quot;&gt;Graphviz&lt;/a&gt;&lt;/h5&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;h5 id=&quot;networkx-python&quot;&gt;&lt;a href=&quot;https://networkx.github.io/&quot;&gt;Networkx (python)&lt;/a&gt;&lt;/h5&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;h5 id=&quot;graphtea-software&quot;&gt;&lt;a href=&quot;http://www.graphtheorysoftware.com/&quot;&gt;GraphTea (software)&lt;/a&gt;&lt;/h5&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;h3 id=&quot;실습&quot;&gt;실습&lt;/h3&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;p&gt;Graphviz&lt;/p&gt;

    &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;from graphviz import Digraph
   
g = Digraph('G', filename='cluster.gv')
   
# NOTE: the subgraph name needs to begin with 'cluster' (all lowercase)
#       so that Graphviz recognizes it as a special cluster subgraph
   
with g.subgraph(name='cluster_0') as c:
    c.attr(style='filled')
    c.attr(color='lightgrey')
    c.node_attr.update(style='filled', color='white')
    c.edges([('a0', 'a1'), ('a1', 'a2'), ('a2', 'a3')])
    c.attr(label='process #1')
   
with g.subgraph(name='cluster_1') as c:
    c.node_attr.update(style='filled')
    c.edges([('b0', 'b1'), ('b1', 'b2'), ('b2', 'b3')])
    c.attr(label='process #2')
    c.attr(color='blue')
   
g.edge('start', 'a0')
g.edge('start', 'b0')
g.edge('a1', 'b3')
g.edge('b2', 'a3')
g.edge('a3', 'a0')
g.edge('a3', 'end')
g.edge('b3', 'end')
   
g.node('start', shape='Mdiamond')
g.node('end', shape='Msquare')
   
g.view()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;    &lt;/div&gt;

    &lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54103595-58b60480-4410-11e9-8be6-38da4a245a49.png&quot; alt=&quot;process_image&quot; /&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Networkx&lt;/p&gt;

    &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;import matplotlib.pyplot as plt
import networkx as nx
import numpy
   
numpy.random.seed(9)
   
G  = nx.grid_2d_graph(4,4)  #4x4 grid
pos= nx.spring_layout(G,iterations=100)
   
plt.subplot(221)
nx.draw(G,pos,node_color='y',font_size=8)
   
plt.subplot(222)
nx.draw(G,pos,node_color='k',node_size=3,with_labels=True)
   
plt.subplot(223)
nx.draw(G,pos,node_color='r',node_size=250,with_labels=False,width=3)
   
plt.subplot(224)
H=G.to_directed()
nx.draw(H,pos,node_color='b',node_size=20,with_labels=False)
   
#plt.savefig(&quot;four_grids.png&quot;)
plt.show()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;    &lt;/div&gt;

    &lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54103608-679cb700-4410-11e9-947f-40d9afc9f7c9.png&quot; alt=&quot;graph_image&quot; /&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;GraphTea
&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54103623-77b49680-4410-11e9-8e03-d7ced2eef544.png&quot; alt=&quot;GraphTea&quot; /&gt;&lt;/p&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;h3 id=&quot;과제&quot;&gt;과제&lt;/h3&gt;

&lt;h5 id=&quot;기하-그래프의-최단거리&quot;&gt;기하 그래프의 최단거리&lt;/h5&gt;

&lt;p&gt;[문제] 어떤 단순 그래프의 특정 두 지점  s 와  t 에 이르는 최단경로를 계산해서 이 경로를 그림으로 표시하는 Python 프로그램과 NetworkX의 알고리즘을 이용해서 작성하시오.&lt;/p&gt;

&lt;p&gt;[입력] 입력파일의 이름은 graph.txt 이다. 입력 파일의 첫 줄에는 2개의 정수 N과 L이 주어진다. N은 정점(vertices)의 개수를 나타내고 L은 두 정점이 연결되는 제한 길이를 표시한다. 이어지는 N개의 줄에는 각 i 번째의 정점 V_i 의 위치좌표 (X_i, Y_i)의 두 정수  X_i, Y_i  가 주어진다. 정점의 번호는 1번부터 2, .. N으로 지정된다. 만일 두 정점  V_i  와  Y_i 의 거리가 L이하면 두 정점은 edge로 연결된다. 즉 아래를 만족하면 edge  (i , j)가 생성된다.&lt;/p&gt;

&lt;script type=&quot;math/tex; mode=display&quot;&gt;\begin{align*}
\sqrt{(x_i-x_j)^2+(y_i-y_j)^2} \leq L
\end{align*}&lt;/script&gt;

&lt;p&gt;여러분은 이렇게 생성된 그래프의 두 정점  s, t 사이의 shortest path를 다른 색으로 확인할 수 있도록 표현하여 
그림으로 visualize해야 한다. 시작점, 도착점  s, t 는 입력파일 graph.txt의 제일 마지막 줄에 주어진다.&lt;/p&gt;

&lt;p&gt;[조건] 입력되는 그래프의 정점의 개수는 10개 이상, 50개 이하이다. L에 따라서 최단거리가 없을 수도 있다.&lt;/p&gt;

&lt;h5 id=&quot;코드&quot;&gt;코드&lt;/h5&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;import matplotlib.pyplot as plt
import networkx as nx
import math

def getFile():
    fp = open(&quot;graph.txt&quot;)
    condition = fp.readline().replace('\n','').split(' ')
    num_vertex, edge_condition = int(condition[0]) , float(condition[1])
    vertexList = []
    for i in range(num_vertex):
        tmp = fp.readline().replace('\n','').split(' ')
        tmp = [int(i) for i in tmp]         
        vertexList.append(tmp)
    target_position = fp.readline().replace('\n', '').split(' ')
    target_position = [int(i) for i in target_position]   
    return edge_condition,vertexList,target_position

def getGraph(graph,vertexList):
    index = 1
    for location in vertexList:
        graph.add_node(index, color='black', pos=location)
        index += 1
    pos = nx.get_node_attributes(graph,'pos')
    return graph, pos

def getDistance(tmp1, tmp2):
    return math.hypot(tmp2[0] - tmp1[0], tmp2[1] - tmp1[1])

def getEdge(vertexList,limitEdge):
    edgeList =[]
    length = len(vertexList)
    for i in range(length-1):
        for j in range(i+1,length):
            dist = getDistance(vertexList[i],vertexList[j])
            if dist &amp;lt; limitEdge:
                edgeList.append((i+1,j+1))
    return edgeList

def main():
    G = nx.Graph()
    limitEdge , vertexList , position = getFile()
    edgeList = getEdge(vertexList, limitEdge)
    G,pos = getGraph(G,vertexList)

    # To add distance between node
    # if not : G.add_edges_from(edgeList , color = 'black') is simple.
    for i in range(len(edgeList)):
        first,second = vertexList[edgeList[i][0]-1], vertexList[edgeList[i][1]-1]
        dist = getDistance(first,second)
        G.add_edge(edgeList[i][0], edgeList[i][1], color='black', label=round(dist,3), weight=1)

    start_Position,end_Position = position[0], position[1]
    shortest_path_length = 0
    if nx.has_path(G,start_Position,end_Position):
        shortestPath = nx.shortest_path(G, start_Position, end_Position)
        for i in range(len(shortestPath) - 1):
            G.add_edge(shortestPath[i], shortestPath[i + 1], color='r', weight=3)
            shortest_path_length += getDistance(vertexList[shortestPath[i]-1],vertexList[shortestPath[i+1]-1])

    edges = G.edges()
    colors = [G[u][v]['color'] for u, v in edges]
    weights = [G[u][v]['weight'] for u, v in edges]
    edge_labels = dict([((u, v,), d['label']) for u, v, d in G.edges(data=True)])
    comment = str(shortest_path_length) if shortest_path_length != 0 else 'None'
    plt.figure(num='LeeHoYong_graph.jpg')                
    nx.draw(G, pos, node_color = 'black', font_color='white', edge_color = colors, width = weights, with_labels=True)
    nx.draw_networkx_edge_labels(G, pos, edge_labels=edge_labels)
    plt.plot(0,0,'r',label='Shortest_Path_Length : ' + comment) 
    plt.legend()
    plt.savefig('LeeHoYong_graph.jpg')      # after installing [ pip install pillow ]
    plt.show()


if __name__ == '__main__':
    main()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h5 id=&quot;결과&quot;&gt;결과&lt;/h5&gt;

&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/23732754/54103645-88650c80-4410-11e9-916c-0785d280ac7d.jpg&quot; alt=&quot;LeeHoYong_graph&quot; /&gt;&lt;/p&gt;

</description>
        <pubDate>Mon, 11 Mar 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/graph/algoithm/np-complete/tool/2019/03/11/%EC%9D%91%EC%9A%A9%EA%B7%B8%EB%9E%98%ED%94%84-week1/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/graph/algoithm/np-complete/tool/2019/03/11/%EC%9D%91%EC%9A%A9%EA%B7%B8%EB%9E%98%ED%94%84-week1/</guid>
        
        <category>graph</category>
        
        <category>algorithm</category>
        
        <category>NP-complete</category>
        
        <category>Tool</category>
        
        
        <category>graph</category>
        
        <category>algoithm</category>
        
        <category>NP-complete</category>
        
        <category>Tool</category>
        
      </item>
    
      <item>
        <title>[BioInfomatics] BUSCO Installation &amp; Usage</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#busco-v3-installation--usage&quot; id=&quot;markdown-toc-busco-v3-installation--usage&quot;&gt;BUSCO v3 Installation &amp;amp; Usage&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#installation&quot; id=&quot;markdown-toc-installation&quot;&gt;Installation&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#input&quot; id=&quot;markdown-toc-input&quot;&gt;Input&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#usage&quot; id=&quot;markdown-toc-usage&quot;&gt;Usage&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#result&quot; id=&quot;markdown-toc-result&quot;&gt;Result&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;busco-v3-installation--usage&quot;&gt;BUSCO v3 Installation &amp;amp; Usage&lt;/h1&gt;

&lt;p&gt;BUSCO &lt;em&gt;v3&lt;/em&gt; is a tool providing quantitative measures for the assessment of genome assembly, gene set, and transcriptome completeness, based on evolutionarily-informed expectations of gene content from near-universal single-copy orthologs selected from &lt;a href=&quot;http://orthodb.org/&quot;&gt;OrthoDB &lt;em&gt;v9&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;BUSCO also provides user guide : &lt;a href=&quot;https://gitlab.com/ezlab/busco/blob/master/BUSCO_v3_userguide.pdf&quot;&gt;BUSCO_v3_userguide.pdf&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;cite : https://busco.ezlab.org&lt;/p&gt;

&lt;h2 id=&quot;installation&quot;&gt;Installation&lt;/h2&gt;

&lt;p&gt;I executed BUSCO v3 on &lt;strong&gt;Ubuntu Server 16.04 LTS&lt;/strong&gt;. (Amazon AWS)&lt;/p&gt;

&lt;blockquote&gt;
  &lt;ol&gt;
    &lt;li&gt;git clone https://gitlab.com/ezlab/busco.git&lt;/li&gt;
    &lt;li&gt;move directory to busco.&lt;/li&gt;
    &lt;li&gt;sudo python setup.py install.&lt;/li&gt;
  &lt;/ol&gt;
&lt;/blockquote&gt;

&lt;p&gt;Installation of BUSCO only is finished.&lt;/p&gt;

&lt;p&gt;But, it needs additional software to use BUSCO v3 :(   (not described in detail in user guide)&lt;/p&gt;

&lt;p&gt;Those are NCBI BLAST+ , HMMER , Augustus.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://i-h1.pinimg.com/564x/6c/99/a0/6c99a0b8d272be67d80c5230ebadae61.jpg&quot; alt=&quot;FPTraffic&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Especially, installation of Augustus is complex. here is a list of dependency to Augustus.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;bamtools, samtools, bcftools, htslib , boost, zlib1g&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And, each tool also have a dependency.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;bamtools - zlib&lt;/p&gt;

  &lt;p&gt;htslib - zlib, liblzma, libbzip2&lt;/p&gt;

  &lt;p&gt;samtools - libncurses5-dev libncursesw5-dev&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;After installing all dependency to Augustus, should change path of software.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Augustus/auxprogs/bam2hints/Makefile&lt;/p&gt;

  &lt;p&gt;Augustus/auxprogs/filterBam/src/Makefile&lt;/p&gt;

  &lt;p&gt;Augustus/auxprogs/bam2wig/Makefile&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Then, you can run Augustus perfectly.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;I'll post easy-to-use install script soon.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h1 id=&quot;input&quot;&gt;Input&lt;/h1&gt;
&lt;blockquote&gt;
  &lt;p&gt;Organism Image	Macaca nemestrina (pig-tailed macaque)
Macaca nemestrina overview&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/956/065/GCF_000956065.1_Mnem_1.0/GCF_000956065.1_Mnem_1.0_genomic.fna.gz&quot;&gt;Download&lt;/a&gt;&lt;/p&gt;

&lt;h1 id=&quot;usage&quot;&gt;Usage&lt;/h1&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;python BUSCO.py -i [SEQUENCE_FILE] -l [LINEAGE] -o [OUTPUT_NAME] -m [MODE] -sp [SPECIES] (-c [CPU count])

 -i FASTA FILE, --in 			FASTA FILE
 -l LINEAGE,    --lineage_path  Specify location of the BUSCO lineage data to be used.
 -o OUTPUT,     --out OUTPUT
 -m MODE,       --mode MODE     Specify which BUSCO analysis mode to run.
                                There are three valid modes:
                                - geno or genome, for genome assemblies 
                                - tran or transcriptome, for transcriptome assemblies 
                                - prot or proteins, for annotated gene sets (protein) 
 -sp SPECIES,   --species SPECIES  
                                Name of existing Augustus species gene finding 
                                parameters. See Augustus documentation for available 
                                options.
 -c N,          --cpu N         Specify the number (N=integer) of threads/cores to use.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h1 id=&quot;result&quot;&gt;Result&lt;/h1&gt;
&lt;p&gt;​								short_summary_result of BUSCO execution&lt;/p&gt;
&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# BUSCO version is: 3.1.0
# The lineage dataset is: mammalia_odb9 (Creation date: 2016-02-13, number of species: 50, number of BUSCOs: 4104)
# To reproduce this run: python busco-master/scripts/run_BUSCO.py -i 
  data/GCF_000956065.1_Mnem_1.0_genomic.fna -o result -l mammalia_odb9/ 
  -m genome -c 16 -sp human

# Summarized benchmarking in BUSCO notation for file data/GCF_000956065.1_Mnem_1.0_genomic.fna
# BUSCO was run in mode: genome

        C:94.7%[S:93.7%,D:1.0%],F:2.1%,M:3.2%,n:4104

        3889    Complete BUSCOs (C)
        3847    Complete and single-copy BUSCOs (S)
        42      Complete and duplicated BUSCOs (D)
        88      Fragmented BUSCOs (F)
        127     Missing BUSCOs (M)
        4104    Total BUSCO groups searched

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;​									full_table_result of BUSCO execution&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# Busco id      Status          Contig        Start           End          Score   Length
EOG090A0FFC   Complete      NW_012014355.1  10381323        10382532        144.0   83
EOG090A0FG2   Complete      NW_012011723.1  598963  642561  190.4   92
EOG090A0FM8   Complete      NW_012012355.1  18069359        18078634        177.3   84
EOG090A0FQ0   Complete      NW_012011278.1  5427743 5447851 159.0   77
EOG090A0FRR   Complete      NW_012012000.1  1219700 1231323 223.9   104
EOG090A0FSN   Complete      NW_012010800.1  1324877 1365806 212.0   99
EOG090A0FSO   Duplicated    NW_012014022.1  22365084        22381425        168.5   89
EOG090A0FSO   Duplicated    NW_012018132.1  13035685        13051851        146.6   88
EOG090A0FTC   Complete      NW_012011100.1  1044303 1045003 187.3   80
EOG090A0FUE   Complete      NW_012012244.1  17031932        17032222        147.5   78
EOG090A0FUU   Missing
EOG090A0FUY   Complete      NW_012010745.1  1404622 1453453 475.2   296
EOG090A0FVS   Complete      NW_012017243.1  22347808        22367367        147.7   75
EOG090A0FW2   Complete      NW_012018354.1  12533257        12538646        170.3   96
EOG090A0FXI   Complete      NW_012013244.1  14331782        14338455        221.1   104
EOG090A0FY5   Complete      NW_012012799.1  6543474 6553096 160.1   101
EOG090A0GAQ   Complete      NW_012010800.1  36211955        36232182        331.4   251

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

</description>
        <pubDate>Wed, 06 Mar 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/software/bioinfomatics/2019/03/06/BUSCO_v3/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/software/bioinfomatics/2019/03/06/BUSCO_v3/</guid>
        
        <category>software</category>
        
        <category>bioinfomatics</category>
        
        
        <category>software</category>
        
        <category>bioinfomatics</category>
        
      </item>
    
      <item>
        <title>[Introduction to Statistics]</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#explaining-standard-deviation&quot; id=&quot;markdown-toc-explaining-standard-deviation&quot;&gt;Explaining Standard Deviation&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#average&quot; id=&quot;markdown-toc-average&quot;&gt;AVERAGE&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#standard-deviation&quot; id=&quot;markdown-toc-standard-deviation&quot;&gt;STANDARD DEVIATION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#more-on-standard-deviation&quot; id=&quot;markdown-toc-more-on-standard-deviation&quot;&gt;MORE ON STANDARD DEVIATION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#use-of-standard-deviation&quot; id=&quot;markdown-toc-use-of-standard-deviation&quot;&gt;USE OF STANDARD DEVIATION&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#are-the-skewness-and-kurtosis-useful-statistics&quot; id=&quot;markdown-toc-are-the-skewness-and-kurtosis-useful-statistics&quot;&gt;Are the Skewness and Kurtosis Useful Statistics?&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#introduction&quot; id=&quot;markdown-toc-introduction&quot;&gt;INTRODUCTION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#skewness&quot; id=&quot;markdown-toc-skewness&quot;&gt;SKEWNESS&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#kurtosis&quot; id=&quot;markdown-toc-kurtosis&quot;&gt;KURTOSIS&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#our-population&quot; id=&quot;markdown-toc-our-population&quot;&gt;OUR POPULATION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#impact-of-sample-size-on-skewness-and-kurtosis&quot; id=&quot;markdown-toc-impact-of-sample-size-on-skewness-and-kurtosis&quot;&gt;IMPACT OF SAMPLE SIZE ON SKEWNESS AND KURTOSIS&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#conclusions&quot; id=&quot;markdown-toc-conclusions&quot;&gt;CONCLUSIONS&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#normal-distribution&quot; id=&quot;markdown-toc-normal-distribution&quot;&gt;Normal distribution&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#introduction-to-the-normal-distribution&quot; id=&quot;markdown-toc-introduction-to-the-normal-distribution&quot;&gt;INTRODUCTION TO THE NORMAL DISTRIBUTION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#probability-density-function&quot; id=&quot;markdown-toc-probability-density-function&quot;&gt;PROBABILITY DENSITY FUNCTION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#standard-normal-distribution&quot; id=&quot;markdown-toc-standard-normal-distribution&quot;&gt;STANDARD NORMAL DISTRIBUTION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#how-to-use-the-normal-distribution&quot; id=&quot;markdown-toc-how-to-use-the-normal-distribution&quot;&gt;HOW TO USE THE NORMAL DISTRIBUTION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#normal-distribution-and-specifications&quot; id=&quot;markdown-toc-normal-distribution-and-specifications&quot;&gt;NORMAL DISTRIBUTION AND SPECIFICATIONS&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#histograms&quot; id=&quot;markdown-toc-histograms&quot;&gt;Histograms&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#normal-probability-plots&quot; id=&quot;markdown-toc-normal-probability-plots&quot;&gt;NORMAL PROBABILITY PLOTS&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#summary&quot; id=&quot;markdown-toc-summary&quot;&gt;SUMMARY&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#5-anderson-darling-test-for-normality&quot; id=&quot;markdown-toc-5-anderson-darling-test-for-normality&quot;&gt;5. Anderson-Darling Test for Normality&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#the-anderson-darling-test-hypotheses&quot; id=&quot;markdown-toc-the-anderson-darling-test-hypotheses&quot;&gt;THE ANDERSON-DARLING TEST HYPOTHESES&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#two-data-sets&quot; id=&quot;markdown-toc-two-data-sets&quot;&gt;TWO DATA SETS&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#the-anderson-darling-test&quot; id=&quot;markdown-toc-the-anderson-darling-test&quot;&gt;THE ANDERSON-DARLING TEST&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#the-p-value-for-the-adjusted-anderson-darling-statistic&quot; id=&quot;markdown-toc-the-p-value-for-the-adjusted-anderson-darling-statistic&quot;&gt;THE P VALUE FOR THE ADJUSTED ANDERSON-DARLING STATISTIC&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#applying-the-anderson-darling-test&quot; id=&quot;markdown-toc-applying-the-anderson-darling-test&quot;&gt;APPLYING THE ANDERSON-DARLING TEST&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#summary-1&quot; id=&quot;markdown-toc-summary-1&quot;&gt;SUMMARY&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#hypothesis-testing&quot; id=&quot;markdown-toc-hypothesis-testing&quot;&gt;Hypothesis Testing&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#the-problem&quot; id=&quot;markdown-toc-the-problem&quot;&gt;THE PROBLEM&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#a-brief-pause-for-the-standard-normal-distribution&quot; id=&quot;markdown-toc-a-brief-pause-for-the-standard-normal-distribution&quot;&gt;A BRIEF PAUSE FOR THE STANDARD NORMAL DISTRIBUTION&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#step-1-formulate-the-null-hypothesis-and-alternative-hypothesis&quot; id=&quot;markdown-toc-step-1-formulate-the-null-hypothesis-and-alternative-hypothesis&quot;&gt;STEP 1: FORMULATE THE NULL HYPOTHESIS AND ALTERNATIVE HYPOTHESIS&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#step-2-determine-the-significance-level-you-want&quot; id=&quot;markdown-toc-step-2-determine-the-significance-level-you-want&quot;&gt;STEP 2: DETERMINE THE SIGNIFICANCE LEVEL YOU WANT&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#step-3-collect-the-data-and-calculate-the-sample-statistics&quot; id=&quot;markdown-toc-step-3-collect-the-data-and-calculate-the-sample-statistics&quot;&gt;STEP 3: COLLECT THE DATA AND CALCULATE THE SAMPLE STATISTICS&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#step-4-calculate-the-p-value&quot; id=&quot;markdown-toc-step-4-calculate-the-p-value&quot;&gt;STEP 4: CALCULATE THE P VALUE&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#step-5--compare-the-p-value-to-the-desired-significance-level&quot; id=&quot;markdown-toc-step-5--compare-the-p-value-to-the-desired-significance-level&quot;&gt;STEP 5:  COMPARE THE P VALUE TO THE DESIRED SIGNIFICANCE LEVEL&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#summary-2&quot; id=&quot;markdown-toc-summary-2&quot;&gt;SUMMARY&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;explaining-standard-deviation&quot;&gt;Explaining Standard Deviation&lt;/h1&gt;
&lt;hr /&gt;
&lt;p&gt;Have you ever had to explain to anyone what a standard deviation is? Or perhaps, you aren’t sure yourself what it is. What is it used for? Why is it important in process improvement? When using control charts, the standard deviation, as well as the average, is a very important parameter. One must understand what is meant by the standard deviation. This newsletter addresses this. We will start with describing what an average is.&lt;/p&gt;

&lt;h3 id=&quot;average&quot;&gt;AVERAGE&lt;/h3&gt;

&lt;p&gt;The average (also called the mean) is probably well understood by most. It represents a “typical” value. For example, the average temperature for the day based on the past is often given on weather reports. It represents a typical temperature for the time of year.&lt;/p&gt;

&lt;p&gt;The average is calculated by adding up the results you have and dividing by the number of results. For example, suppose we have wire cable that is cut to different lengths for a customer. These lengths, in feet, are 5, 6, 2, 3, and 8. The average is determined by adding up these five numbers and dividing by 5. In this case, the average (X) is:&lt;/p&gt;

&lt;p&gt;​									X = (5+6+2+3+8)/5 = 4.8&lt;/p&gt;

&lt;p&gt;The average length of wire for these five pieces is 4.8 feet.&lt;/p&gt;

&lt;h3 id=&quot;standard-deviation&quot;&gt;STANDARD DEVIATION&lt;/h3&gt;

&lt;p&gt;While the average is understood by most, the standard deviation is understood by few. To begin to understand what a standard deviation is, consider the two histograms. Histogram 1 has more variation than Histogram 2.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/morevar.gif&quot; alt=&quot;sample histogram 1&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/lessvar.gif&quot; alt=&quot;sample histogram 2&quot; /&gt;&lt;/p&gt;

&lt;p&gt;In the first histogram, the largest value is 9, while the smallest value is 1. The overall range of data is 9 - 1 = 8. In the second histogram, the overall range is 7 - 3 = 4. The range is larger for Histogram 1. Ranges are often used in control charts for variation (for example, the X-R charts). In fact, the average range from a control chart can be used to calculate the process standard deviation.&lt;/p&gt;

&lt;p&gt;The average of the data in each histogram is 5. So, in this case, the highest bar is the average. We can also see that Histogram 1 has more variation than Histogram 2 because the distance, on average, of the individual observations from the overall average (5) is greater in Histogram 1 than Histogram 2. This distance is usually referred to as a deviation. If your have a result X = 3, the deviation of this value from the average is 3 - 5 = - 2 or the value “3” is two units below the average.&lt;/p&gt;

&lt;p&gt;One can view the standard deviation as being an “average” distance each individual measurement is from the average, X. Let’s return to the numbers we found the average for earlier to see how we can estimate this average deviation from X. These numbers were the length of wire cable we had cut. We want to find the average distance each number is from X = 4.8. To do this, we can determine the deviation of each number from the average as shown below.&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;&lt;strong&gt;Length (X)&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Deviation from Average (X- X)&lt;/strong&gt;&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;5&lt;/td&gt;
      &lt;td&gt;5 - 4.8 = 0.2&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;6&lt;/td&gt;
      &lt;td&gt;6 - 4.8 = 1.2&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2&lt;/td&gt;
      &lt;td&gt;2 - 4.8 = -2.8&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3&lt;/td&gt;
      &lt;td&gt;3 - 4.8 = -1.8&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;8&lt;/td&gt;
      &lt;td&gt;8 - 4.8 = 3.2&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;If we add up the deviations from average, we discover that:&lt;/p&gt;

&lt;p&gt;Sum of the deviations from average = 0.2 + 1.2 + (- 2.8) + (-1.8) + 3.2 = 0&lt;/p&gt;

&lt;p&gt;The sum of the deviations from average added up to zero! This is not a coincidence. The fact is that when the deviations from the average are added up, the sum will always be zero if the average was calculated from the data. We cannot use this method to determine the standard deviation. The negative signs in the sum of deviations are what cause the sum to be zero. Suppose instead that we square each deviation from the average (i.e., multiply the deviation by itself). Squaring a negative number makes it positive. In this case the square of the &lt;strong&gt;deviations are:&lt;/strong&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;&lt;strong&gt;Length (X)&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Square of Deviation from Average (X-X)2&lt;/strong&gt;&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;5&lt;/td&gt;
      &lt;td&gt;(5 - 4.8)2 = (0.2)2 = .04&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;6&lt;/td&gt;
      &lt;td&gt;(6 - 4.8)2 = (1.2)2 = 1.44&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2&lt;/td&gt;
      &lt;td&gt;(2 - 4.8)2 = (-2.8)2 = 7.84&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3&lt;/td&gt;
      &lt;td&gt;(3 - 4.8)2 = (-1.8)2 = 3.24&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;8&lt;/td&gt;
      &lt;td&gt;(8 - 4.8)2 = (3.2)2 = 10.24&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;h3 id=&quot;more-on-standard-deviation&quot;&gt;MORE ON STANDARD DEVIATION&lt;/h3&gt;

&lt;p&gt;The sum of these squares of deviations from the average is 22.8. This number can now be used to determine the “average” distance each individual result is from X. The temptation here is to divide by n = 5 since there are five lengths. Unfortunately, this would be incorrect. The reason is that we would actually be underestimating the true standard deviation. This is primarily because we used the data to calculate an average (we don’t know the true average of the process). This means that there are only n-1 independent pieces of information. If we know the average and four of the individual results, the fifth result can be determined. Thus, the correct number to divide by is n - 1 = 4.&lt;/p&gt;

&lt;p&gt;Thus, the sum of the squares of the deviation from the average divided by 4 is 22.8/4 = 5.7. Remember, this number contains the squares of the deviations. To get to the standard deviation, we must take the square root of that number. Thus, the standard deviation is square root of 5.7 = 2.4.&lt;/p&gt;

&lt;p&gt;The equation for a sample standard deviation we just calculated is shown in the figure. Control charts are used to estimate what the process standard deviation is. For example, the average range on the X-R chart can be used to estimate the standard deviation using the equation s = R/d2 where d2 is a control chart constant (see March 2005 newsletter).&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/sigmaequation.gif&quot; alt=&quot;sigma equation&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;use-of-standard-deviation&quot;&gt;USE OF STANDARD DEVIATION&lt;/h3&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/nd.gif&quot; alt=&quot;normal distribution diagram&quot; /&gt;&lt;/p&gt;

&lt;p&gt;So, why are the standard deviation and, for that matter, the average important? There are many types of control charts that are based on variables data. The basic probability distribution that governs the calculation of the control limits on these charts is the normal distribution. The normal distribution is the familiar bell shaped curve shown in the above figure.&lt;/p&gt;

&lt;p&gt;The normal distribution has several interesting characteristics. The shape of the distribution is determined by the average, X, and the standard deviation, s. The highest point on the curve is the average. The distribution is symmetrical about the average. Most of the area under the curve (99.7%) lies between -3s and +3s of the average. In addition, about 95.44% of the curve is between -2s and +2s of the average, while 68.26% of the curve is between -1s and +1s of the average.&lt;/p&gt;

&lt;p&gt;To determine if a normal distribution exists, you can make a histogram of the individual results. If this histogram is bell-shaped, you can assume that the individual measurements are normally distributed. For example, suppose you are monitoring how long it takes to approve or disapprove a credit application for a customer. From your control charts (assume the process is in control), you have estimated the process average to be 14 working days and the standard deviation to be 2 days. After constructing a histogram on the days to approve or disapprove a credit application, you discover that it is bell-shaped. You can now replace the histogram with the normal distribution shown in the figure below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/ndnum.gif&quot; alt=&quot;normal distribution diagram 2&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Since the process is in statistical control, you know that about 67% of the time, it will take 12 to 16 days to process a credit application; 95% of the time it will take 10 to 18 days; and 99.7% of the time it will take 8 to 20 days. So, the average and standard deviation (for a normal distribution) allow you to begin to consider process capability. Can the process meet our customer specifications? Take a look at our three part newsletter on process capability.&lt;/p&gt;

&lt;hr /&gt;
&lt;h1 id=&quot;are-the-skewness-and-kurtosis-useful-statistics&quot;&gt;Are the Skewness and Kurtosis Useful Statistics?&lt;/h1&gt;
&lt;hr /&gt;

&lt;p&gt;You have a set of samples.  Maybe you took 15 samples from a batch of finished product and measured those samples for density.  Now you are armed with data you can analyze.  And your software package has a feature that will generate the descriptive statistics for these data.  You enter the data into your software package and run the descriptive statistics.  You get a lot of numbers – the sample size, average, standard deviation, range, maximum, minimum and a host of other numbers.  You spy two numbers: the skewness and kurtosis.  What do these two statistics tell you about your sample?&lt;/p&gt;

&lt;p&gt;This month’s publication covers the skewness and kurtosis statistics.  These two statistics are called “shape” statistics, i.e., they describe the shape of the distribution. What do the skewness and kurtosis really represent? And do they help you understand your process any better? Are they useful statistics? Let’s take a look&lt;/p&gt;

&lt;p&gt;You may download a pdf copy of this publication &lt;a href=&quot;https://www.spcforexcel.com/publications/are-skewness-and-kurtosis-useful-statistics/index.html&quot;&gt;at this link&lt;/a&gt;.  You may also download an Excel workbook containing the impact of sample size on skewness and kurtosis at the end of this publication.  You may also leave a comment at the end of the publication.&lt;/p&gt;

&lt;h3 id=&quot;introduction&quot;&gt;INTRODUCTION&lt;/h3&gt;

&lt;p&gt;Skewness and kurtosis are two commonly listed values when you run a software’s descriptive statistics function.  Many books say that these two statistics give you insights into the shape of the distribution.&lt;/p&gt;

&lt;p&gt;Skewness is a measure of the symmetry in a distribution.  A symmetrical dataset will have a skewness equal to 0.  So, a normal distribution will have a skewness of 0.   Skewness essentially measures the relative size of the two tails.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/histogram%20nd.png&quot; alt=&quot;histogram&quot; /&gt;Kurtosis is a measure of the combined sizes of the two tails.  It measures the amount of probability in the tails.  The value is often compared to the kurtosis of the normal distribution, which is equal to 3.  If the kurtosis is greater than 3, then the dataset has heavier tails than a normal distribution (more in the tails).  If the kurtosis is less than 3, then the dataset has lighter tails than a normal distribution (less in the tails).  Careful here.  Kurtosis is sometimes reported as “excess kurtosis.”  Excess kurtosis is determined by subtracting 3 form the kurtosis.  This makes the normal distribution kurtosis equal 0.  Kurtosis originally was thought to measure the peakedness of a distribution.   Though you will still see this as part of the definition in many places, this is a misconception.&lt;/p&gt;

&lt;p&gt;Skewness and kurtosis involve the tails of the distribution.  These are presented in more detail below.&lt;/p&gt;

&lt;h3 id=&quot;skewness&quot;&gt;SKEWNESS&lt;/h3&gt;

&lt;p&gt;Skewness is usually described as a measure of a dataset’s symmetry – or lack of symmetry.   A perfectly symmetrical data set will have a skewness of 0.   The normal distribution has a skewness of 0.&lt;/p&gt;

&lt;p&gt;The skewness is defined as (Advanced Topics in Statistical Process Control, Dr. Donald Wheeler, &lt;a href=&quot;http://www.spcpress.com/&quot;&gt;www.spcpress.com&lt;/a&gt;):&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/a3.png&quot; alt=&quot;skewness equation&quot; /&gt;&lt;/p&gt;

&lt;p&gt;where n is the sample size, Xi is the ith X value, X is the average and s is the sample standard deviation.  Note the exponent in the summation.  It is “3”.  The skewness is referred to as the “third standardized central moment for the probability model.”&lt;/p&gt;

&lt;p&gt;Most software packages use a formula for the skewness that takes into account sample size:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/skewness-above-below.png&quot; alt=&quot;skewness equation&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This sample size formula is used here.  It is also what Microsoft Excel uses.  The difference between the two formula results becomes very small as the sample size increases.&lt;/p&gt;

&lt;p&gt;Figure 1 is a symmetrical data set.  It was created by generating a set of data from 65 to 135 in steps of 5 with the number of each value as shown in Figure 1.  For example, there are 3 65’s, 6 75’s, etc.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 1: Symmetrical Dataset with Skewness = 0&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/Figure-1.png&quot; alt=&quot;symmetrical distribution&quot; /&gt;&lt;/p&gt;

&lt;p&gt;A truly symmetrical data set has a skewness equal to 0.  It is easy to see why this is true from the skewness formula. Look at the term in the numerator after the summation sign.  Each individual X value is subtracted from the average.  So, if a set of data is truly symmetrical, for each point that is a distance “d” above the average, there will be a point that is a distance “-d” below the average.&lt;/p&gt;

&lt;p&gt;Consider the value of 65 and value of 135.  The average of the data in Figure 1 is 100.  So, the following is true when X = 65:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/negative-skewness-equation.png&quot; alt=&quot;negative skewness&quot; /&gt;&lt;/p&gt;

&lt;p&gt;For X = 135, the following is true:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/positive%20skewness%20equation.png&quot; alt=&quot;positive skewness&quot; /&gt;&lt;/p&gt;

&lt;p&gt;So, the -4278 and +4278 even out at 0.  So, a truly symmetrical data set will have a skewness of 0.&lt;/p&gt;

&lt;p&gt;To explore positive and negative values of skewness, let’s define the following term:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/sabove-sbelow.png&quot; alt=&quot;skewness above and below&quot; /&gt;&lt;/p&gt;

&lt;p&gt;So, Sabove can be viewed as the “size” of the deviations from average when Xi is above the average.    Likewise, Sbelow can be viewed as the “size” of the deviations from average when Xi is below the average.  Then, skewness becomes the following:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/skewness-above-below.png&quot; alt=&quot;skewness equation 2&quot; /&gt;&lt;/p&gt;

&lt;p&gt;If Sabove is larger than Sbelow, then skewness will be positive.   This typically means that the right-hand tail will be longer than the left-hand tail.  Figure 2 is an example of this.  The skewness for this dataset is 0.514.  A positive skewness indicates that the size of the right-handed tail is larger than the left-handed tail.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 2: Dataset with Positive Skewness&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/Figure-2.png&quot; alt=&quot;distribution with positive skewness&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Figure 3 is an example of dataset with negative skewness.  It is the mirror image essentially of Figure 2.  The skewness is -0.514.   In this case, Sbelow is larger than Sabove.  The left-hand tail will typically be longer than the right-hand tail.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 3: Dataset with Negative Skewness&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/Figure-3.png&quot; alt=&quot;distribution with negative skewness&quot; /&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;So, when is the skewness too much?  The rule of thumb seems to be:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;If the skewness is between -0.5 and 0.5, the data are fairly symmetrical&lt;/li&gt;
  &lt;li&gt;If the skewness is between -1 and – 0.5 or between 0.5 and 1, the data are moderately skewed&lt;/li&gt;
  &lt;li&gt;If the skewness is less than -1 or greater than 1, the data are highly skewed&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;kurtosis&quot;&gt;KURTOSIS&lt;/h3&gt;

&lt;p&gt;How to define kurtosis?  This is really the reason this article was updated.  If you search for definitions of kurtosis, you will see some definitions that includes the word “peakedness” or other similar terms.  For example,&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;“Kurtosis is the degree of peakedness of a distribution” – &lt;a href=&quot;http://mathworld.wolfram.com/Kurtosis.html&quot;&gt;Wolfram MathWorld&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;“We use kurtosis as a measure of peakedness (or flatness)” – &lt;a href=&quot;http://www.real-statistics.com/descriptive-statistics/symmetry-skewness-kurtosis/&quot;&gt;Real Statistics Using Excel&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;You can find other definitions that include peakedness or flatness when you search the web.  The problem is these definitions are not correct.  Dr. Peter Westfall published an article that addresses why kurtosis does not measure peakedness (&lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4321753/&quot;&gt;link to article&lt;/a&gt;).  He said:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;“Kurtosis tells you virtually nothing about the shape of the peak – its only unambiguous interpretation is in terms of tail extremity.”&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Dr. Westfall includes numerous examples of why you cannot relate the peakedness of the distribution to the kurtosis.&lt;/p&gt;

&lt;p&gt;Dr. Donald Wheeler also discussed this in his&lt;a href=&quot;http://www.qualitydigest.com/inside/quality-insider-article/problems-skewness-and-kurtosis-part-two.html&quot;&gt; two-part series on skewness and kurtosis&lt;/a&gt;.  He said:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/man-woman-ipad.png&quot; alt=&quot;looking at ipad&quot; /&gt; “Kurtosis was originally thought to be a measure the “peakedness” of a distribution.  However, since the central portion of the distribution is virtually ignored by this parameter, kurtosis cannot be said to measure peakedness directly.  While there is a correlation between peakedness and kurtosis, the relationship is an indirect and imperfect one at best.”&lt;/p&gt;

&lt;p&gt;Dr. Wheeler defines kurtosis as:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;“The kurtosis parameter is a measure of the combined weight of the tails relative to the rest of the distribution.”&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;So, kurtosis is all about the tails of the distribution – not the peakedness or flatness.  It measures the tail-heaviness of the distribution.&lt;/p&gt;

&lt;p&gt;Kurtosis is defined as:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/a4.png&quot; alt=&quot;kurtosis equation&quot; /&gt;&lt;/p&gt;

&lt;p&gt;where n is the sample size, Xi is the ith X value, X is the average and s is the sample standard deviation.  Note the exponent in the summation.  It is “4”.  The kurtosis is referred to as the “fourth standardized central moment for the probability model.”&lt;/p&gt;

&lt;p&gt;Here is where is gets a little tricky.  If you use the above equation, the kurtosis for a normal distribution is 3.  Most software packages (including Microsoft Excel) use the formula below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/kurtosis-equation.png&quot; alt=&quot;sample kurtosis equation&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This formula does two things.  It takes into account the sample size and it subtracts 3 from the kurtosis.  With this equation, the kurtosis of a normal distribution is 0.  This is really the excess kurtosis, but most software packages refer to it as simply kurtosis.   The last equation is used here.  So, if a dataset has a positive kurtosis, it has more in the tails than the normal distribution.  If a dataset has a negative kurtosis, it has less in the tails than the normal distribution.&lt;/p&gt;

&lt;p&gt;Since the exponent in the above is 4, the term in the summation will always be positive – regardless of whether Xi is above or below the average.  Xi values close to the average contribute very little to the kurtosis.    The tail values of Xi contribute much more to the kurtosis.&lt;/p&gt;

&lt;p&gt;Look back at Figures 2 and 3.  They are essentially mirror images of each other.  The skewness of these datasets is different: 0.514 and -0.514.  But the kurtosis is the same.  Both have a kurtosis of -0.527.  This is because kurtosis looks at the combined size of the tails.&lt;/p&gt;

&lt;p&gt;The kurtosis decreases as the tails become lighter.  It increases as the tails become heavier.  Figure 4 shows an extreme case.  In this dataset, each value occurs 10 times.  The values are 65 to 135 in increments of 5.  The kurtosis of this dataset is -1.21.  Since this value is less than 0, it is considered to be a “light-tailed” dataset.  It has as much data in each tail as it does in the peak.  Note that this is a symmetrical distribution, so the skewness is zero.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 4: Negative Kurtosis Example&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/Figure-4.png&quot; alt=&quot;distribution with negative kurtosis&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Figure 5 is shows a dataset with more weight in the tails.  The kurtosis of this dataset is 1.86.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 5: Positive Kurtosis Example&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/Figure-5.png&quot; alt=&quot;distribution with positive kurtosis&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Most often, kurtosis is measured against the normal distribution.  If the kurtosis is close to 0, then a normal distribution is often assumed.  These are called mesokurtic distributions.  If the kurtosis is less than zero, then the distribution is light tails and is called a platykurtic distribution.  If the kurtosis is greater than zero, then the distribution has heavier tails and is called a leptokurtic distribution.&lt;/p&gt;

&lt;p&gt;The problem with both skewness and kurtosis is the impact of sample size.  This is described below.&lt;/p&gt;

&lt;h3 id=&quot;our-population&quot;&gt;OUR POPULATION&lt;/h3&gt;

&lt;p&gt;Are the skewness and kurtosis any value to you?  You take a sample from your process and look at the calculated values for the skewness and kurtosis.  What can you tell from these two results?  To explore this, a data set of 5000 points was randomly generated. The goal was to have a mean of 100 and a standard deviation of 10. The random generation resulted in a data set with a mean of 99.95 and a standard deviation of 10.01. The histogram for these data is shown in Figure 6 and looks fairly bell-shaped.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 6: Population Histogram&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/figure-6.png&quot; alt=&quot;population histogram&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/data.png&quot; alt=&quot;data word&quot; /&gt;The skewness of the data is 0.007. The kurtosis is 0.03. Both values are close to 0 as you would expect for a normal distribution. These two numbers represent the “true” value for the skewness and kurtosis since they were calculated from all the data. In real life, you don’t know the real skewness and kurtosis because you have to sample the process. This is where the problem begins for skewness and kurtosis. Sample size has a big impact on the results.&lt;/p&gt;

&lt;h3 id=&quot;impact-of-sample-size-on-skewness-and-kurtosis&quot;&gt;IMPACT OF SAMPLE SIZE ON SKEWNESS AND KURTOSIS&lt;/h3&gt;

&lt;p&gt;The 5,000-point dataset above was used to explore what happens to skewness and kurtosis based on sample size. For example, suppose we wanted to determine the skewness and kurtosis for a sample size of 5.  5 results were randomly selected from the data set above and the two statistics calculated. This was repeated for the sample sizes shown in Table 1.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 1: Impact of Sample Size on Skewness and Kurtosis&lt;/strong&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;&lt;strong&gt;Sample Size&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Skewness&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Kurtosis&lt;/strong&gt;&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;5&lt;/td&gt;
      &lt;td&gt;1.983&lt;/td&gt;
      &lt;td&gt;3.974&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;10&lt;/td&gt;
      &lt;td&gt;-0.078&lt;/td&gt;
      &lt;td&gt;-1.468&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;15&lt;/td&gt;
      &lt;td&gt;-0.384&lt;/td&gt;
      &lt;td&gt;0.127&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;25&lt;/td&gt;
      &lt;td&gt;-0.356&lt;/td&gt;
      &lt;td&gt;-0.025&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;50&lt;/td&gt;
      &lt;td&gt;-0.169&lt;/td&gt;
      &lt;td&gt;-0.752&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;75&lt;/td&gt;
      &lt;td&gt;-0.489&lt;/td&gt;
      &lt;td&gt;0.615&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;100&lt;/td&gt;
      &lt;td&gt;-0.346&lt;/td&gt;
      &lt;td&gt;0.671&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;250&lt;/td&gt;
      &lt;td&gt;0.089&lt;/td&gt;
      &lt;td&gt;0.061&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;500&lt;/td&gt;
      &lt;td&gt;0.186&lt;/td&gt;
      &lt;td&gt;0.232&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;750&lt;/td&gt;
      &lt;td&gt;-0.02&lt;/td&gt;
      &lt;td&gt;0.042&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;1000&lt;/td&gt;
      &lt;td&gt;-0.138&lt;/td&gt;
      &lt;td&gt;0.062&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;1250&lt;/td&gt;
      &lt;td&gt;0.085&lt;/td&gt;
      &lt;td&gt;0.079&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;1500&lt;/td&gt;
      &lt;td&gt;-0.017&lt;/td&gt;
      &lt;td&gt;0.001&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2000&lt;/td&gt;
      &lt;td&gt;-0.059&lt;/td&gt;
      &lt;td&gt;-0.009&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2500&lt;/td&gt;
      &lt;td&gt;0.037&lt;/td&gt;
      &lt;td&gt;0.096&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3000&lt;/td&gt;
      &lt;td&gt;0.009&lt;/td&gt;
      &lt;td&gt;0.005&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3500&lt;/td&gt;
      &lt;td&gt;-0.015&lt;/td&gt;
      &lt;td&gt;0.004&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;4000&lt;/td&gt;
      &lt;td&gt;-0.015&lt;/td&gt;
      &lt;td&gt;-0.009&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;4500&lt;/td&gt;
      &lt;td&gt;0.009&lt;/td&gt;
      &lt;td&gt;0.036&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;5000&lt;/td&gt;
      &lt;td&gt;0.007&lt;/td&gt;
      &lt;td&gt;0.03&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;Notice how much different the results are when the sample size is small compared to the “true” skewness and kurtosis for the 5,000 results. For a sample size of 25, the skewness was -.356 compared to the true value of 0.007 while the kurtosis was -0.025. Both signs are opposite of the true values which would lead to wrong conclusions about the shape of the distribution. There appears to be a lot of variation in the results based on sample size.&lt;/p&gt;

&lt;p&gt;Figure 7 shows how the skewness changes with sample size.  Figure 8 is the same but for kurtosis.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 7: Skewness versus Sample Size&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/Figure-7.png&quot; alt=&quot;skewness versus sample size&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 8: Kurtosis versus Sample Size&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/Skewness-Kurtosis-Figures/Figure-8.png&quot; alt=&quot;kurtosis versus sample size&quot; /&gt;&lt;/p&gt;

&lt;p&gt;30 is a common number of samples used for process capability studies. A subgroup size of 30 was randomly selected from the data set. This was repeated 100 times. The skewness varied from -1.327 to 1.275 while the kurtosis varied from -1.12 to 2.978. What kind of decisions can you make about the shape of the distribution when the skewness and kurtosis vary so much? Essentially, no decisions.&lt;/p&gt;

&lt;h3 id=&quot;conclusions&quot;&gt;CONCLUSIONS&lt;/h3&gt;

&lt;p&gt;The skewness and kurtosis statistics appear to be very dependent on the sample size. The table above shows the variation. In fact, even several hundred data points didn’t give very good estimates of the true kurtosis and skewness. Smaller sample sizes can give results that are very misleading. Dr Wheeler wrote in his book mentioned above:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;“In short, skewness and kurtosis are practically worthless. Shewhart made this observation in his first book. The statistics for skewness and kurtosis simply do not provide any useful information beyond that already given by the measures of location and dispersion.”&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Walter Shewhart was the “Father” of SPC. So, don’t put much emphasis on skewness and kurtosis values you may see. And remember, the more data you have, the better you can describe the shape of the distribution.  But, in general, it appears there is little reason to pay much attention to skewness and kurtosis statistics.  Just look at the histogram.  It often gives you all the information you need.&lt;/p&gt;

&lt;p&gt;To download the workbook containing the macro and results that generated the above tables, please &lt;a href=&quot;https://www.spcforexcel.com/files/Data-for-Kurtosis-and-Skewness.xls&quot;&gt;click here&lt;/a&gt;.&lt;/p&gt;

&lt;hr /&gt;
&lt;h1 id=&quot;normal-distribution&quot;&gt;Normal distribution&lt;/h1&gt;
&lt;hr /&gt;
&lt;h3 id=&quot;introduction-to-the-normal-distribution&quot;&gt;INTRODUCTION TO THE NORMAL DISTRIBUTION&lt;/h3&gt;

&lt;p&gt;If you search for “normal distribution” on Google, you will get a lot of hits. Wikipedia, the free encyclopedia, starts out its normal distribution with:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;“In probability theory and statistics, the normal distribution or Gaussian distribution is a continuous probability distribution that describes data that clusters around a mean or average. The graph of the associated probability density function is bell-shaped, with a peak at the mean, and is known as the Gaussian function or bell curve.”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;So, what does this mean to us and how do we use normal distributions? This month’s newsletter examines the normal distribution.&lt;/p&gt;

&lt;p&gt;The normal distribution is shown below. It is shaped like a bell&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/nd.png&quot; alt=&quot;normal distribution&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The normal distribution has several interesting characteristics:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;The shape of the distribution is determined by the average, μ (or X), and the standard deviation, σ.&lt;/li&gt;
  &lt;li&gt;The highest point on the curve is the average.&lt;/li&gt;
  &lt;li&gt;The distribution is symmetrical about the average.&lt;/li&gt;
  &lt;li&gt;As you move away from the average, the points occur with less frequency.&lt;/li&gt;
  &lt;li&gt;Most of the area under the curve (99.7%) lies between -3σ and +3σ of the average.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;probability-density-function&quot;&gt;PROBABILITY DENSITY FUNCTION&lt;/h3&gt;

&lt;p&gt;The probability density function for the normal distribution is given by:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/ndprobdensityfunction.png&quot; alt=&quot;normal distribution probabilty function&quot; /&gt;&lt;/p&gt;

&lt;p&gt;where x is some value between -∞ to +∞, μ is the average and σ is the standard deviation. The standard deviation is an indication of how wide the normal distribution is. The average gives the location of the normal distribution.&lt;/p&gt;

&lt;p&gt;The distributions below show how the normal distribution changes as the standard deviation changes. The average is 100 and there are three different distributions with standard deviations of 5, 10, and 20. Note that the larger the standard deviation, the wider the distribution. When you are making a control chart, the range chart is actually monitoring the “width” of the distribution. The range chart answers the following question: is the spread in my data staying the same over time (in control) or is the spread getting smaller or larger (out of control)?&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/threenormaldistributions.png&quot; alt=&quot;three normal distributions with different standard deviations&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The distributions above were generated using Excel’s NORMSDIST function using an average of 100, one of the three standard deviations above and an X values range from 20 to 180. So, if you know your process average and process standard deviation, you can easily draw the normal distribution for your process. This, of course, assumes that your process is normally distributed.&lt;/p&gt;

&lt;h3 id=&quot;standard-normal-distribution&quot;&gt;STANDARD NORMAL DISTRIBUTION&lt;/h3&gt;

&lt;p&gt;For an average of 0 and a standard deviation of 1, the formula above becomes:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/standardnormalfunction.png&quot; alt=&quot;standard normal function&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This is known as the standard normal distribution. For this distribution, the area under the curve from -∞ to +∞ is equal to 1.0. In addition, the area under the curve is proportional to the fraction of measurements that fall in that region. These two facts will be used to help determine the fraction of measurements that fall above some value (such as a specification limit), below some value, or between two values. The standard normal distribution is shown below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/standardnormaldistribution.png&quot; alt=&quot;standard normal distribution&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;how-to-use-the-normal-distribution&quot;&gt;HOW TO USE THE NORMAL DISTRIBUTION&lt;/h3&gt;

&lt;p&gt;Suppose you are interested in a certain quality characteristic, X. You have been monitoring this characteristic using an X-R chart. Both the X chart and the range chart are in statistical control. This means that you can predict what your process will make in the near future. You also know that you have good estimates of the process average (from the X chart) and the process standard deviation (from the range chart). Suppose that X = 100 and that σ = 10.&lt;/p&gt;

&lt;p&gt;In addition, you have constructed a histogram for the last month’s data. The histogram is shown below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/histogram_nd.png&quot; alt=&quot;histogram of data&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This histogram appears to be bell-shaped so you assume that you are dealing with a normal distribution. You can then draw the normal distribution for this process because you know the average and standard deviation (from your control charts). The normal distribution for this process is shown below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/nd_with_percentages.png&quot; alt=&quot;normal distribution with percentages&quot; /&gt;&lt;/p&gt;

&lt;p&gt;A normal distribution has the following properties:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;68% of the data is within +/- 1 standard deviation of the average&lt;/li&gt;
  &lt;li&gt;95% of the data is within +/- 2 standard deviations of the average&lt;/li&gt;
  &lt;li&gt;99.7% of the data is within +/- 3 standard deviations of the average&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For your process, the following calculations can be done:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/ndcalculations.png&quot; alt=&quot;normal distribution calculations: adding sigma&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Thus, 68% of the data lies between 90 and 110; 95% of the data between 80 and 120; and 99.7% of the data between 70 and 130. The specifications for the process have been 65 to 140. Life is good - everything has been within specifications.&lt;/p&gt;

&lt;h3 id=&quot;normal-distribution-and-specifications&quot;&gt;NORMAL DISTRIBUTION AND SPECIFICATIONS&lt;/h3&gt;

&lt;p&gt;Now suppose a customer has decided that the upper specification limit for your process should be 112. You can easily see from the histogram and the normal distribution that some of your product will be out of specification. The question is how much will be out of specification. This is where the z value becomes important. z is defined by the equation:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/zvalueformula.png&quot; alt=&quot;z value formula&quot; /&gt;&lt;/p&gt;

&lt;p&gt;For our example, x = 112, so&lt;/p&gt;

&lt;p&gt;​								z = (112-100)/10 = 12/10 = 1.2&lt;/p&gt;

&lt;p&gt;z represents the number of standard deviations some value is away from the average. So, 112 is 1.2 standard deviations above the average. If z is negative, it means that the value is below the average.&lt;/p&gt;

&lt;p&gt;To find out how much product is more than 1.2 standard deviations above the average, you can use what is called the “z table.” The z table gives the fraction of process output that is beyond some value x that is z standard deviations from the average. The z table is given below.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;z Table: Standard Normal Distribution&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/ztables.png&quot; alt=&quot;z table&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The table above returns a value of .1151 for a value of z = 1.2. This means that 11.51% of the data will be above the upper specification limit of 112. This is shown in the figure below. The area in red represents material that is out of specifications on the high side.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/percentabove.png&quot; alt=&quot;area above usl&quot; /&gt;&lt;/p&gt;

&lt;p&gt;You can also use the NORMSDIST function in Excel to find the above result. This function in Excel returns the fraction of results less than a value. So, you must use 1 - NORMSDIST(z) if you want the fraction of data above z. In this example: 1- NORMSDIST(1.2) = .11507&lt;/p&gt;

&lt;p&gt;To make matters worse, the customer has now set the lower specification as 83. You want to find out what percent of the product will be out of specification on the low side. The process is the same as above. In this case:&lt;/p&gt;

&lt;p&gt;​								z = (83 - 100)/10 = -17/10 = -1.7&lt;/p&gt;

&lt;p&gt;This means that 83 is 1.7 standard deviations below the average. From the z tables, the fraction of data below 1.7 is 0.0446 or 4.46%. In Excel, NORMSDIST(-1.7) = 0.044565. The figure below shows this situation. The area in red represents the product that is below the lower specification limit.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/percentbelow.png&quot; alt=&quot;percent below lsl&quot; /&gt;&lt;/p&gt;

&lt;p&gt;To find out the area between two values, you use the fact that the area under the standard normal curve is 1. For example, suppose you want to find out the area between 83 and 112. We know that the area below 83 is .0446 and the area above 112 is .1151. Then the area between 83 and 112 is given by:&lt;/p&gt;

&lt;p&gt;​								1 - .0446 - .1151 = 0.8403&lt;/p&gt;

&lt;p&gt;Thus, 84.03% of the data lies between 83 and 112. Thus, with the new specifications, 84% of your product will be within specification. The remaining 16% will be out specification. And since the process is in statistical control, this will continue to be true until the process is changed fundamentally.&lt;/p&gt;

&lt;hr /&gt;
&lt;h1 id=&quot;histograms&quot;&gt;Histograms&lt;/h1&gt;
&lt;hr /&gt;
&lt;p&gt;There are a number of ways to determine if you have a normal distribution. One of the easiest is to construct a histogram based on the data. Simply examine the histogram and see if you think it is bell shaped. If you have lots of data, this is a perfectly valid way of determining if your data are normally distributed. Please see our December 2005 and January 2006 newsletters for more information on creating and using histograms.&lt;/p&gt;

&lt;p&gt;Note that a histogram of real data will not look like a perfect normal distribution. All you are trying to determine is if describing the data as a normal distribution is reasonable. For example, take a look at the histogram below. Does it look like a bell-shaped curve? Does it look normal? It is not perfect, but it appears that it is reasonable to assume that these data come from a normal distribution.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/hist_normal.png&quot; alt=&quot;histogram: normal distribution&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Now examine the histogram below. Does it look like a bell-shaped curve? This does not look bell-shaped. Most values tend toward zero. With these data, it is not reasonable to assume that there is a normal distribution present.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/hist_nonnormal.png&quot; alt=&quot;histogram - nonnormal data&quot; /&gt;&lt;/p&gt;

&lt;p&gt;So, it is perfectly valid to use a histogram to determine it you think your data can be reasonably represented by a normal distribution. If you don’t have a lot of data, histograms will not be very useful in determining if you have a normal distribution. You can randomly take 20 samples from a normal distribution and the resulting histogram may not look normal. In these cases, you need to use the normal probability plot.&lt;/p&gt;

&lt;h3 id=&quot;normal-probability-plots&quot;&gt;NORMAL PROBABILITY PLOTS&lt;/h3&gt;

&lt;p&gt;A normal probability plot can be used to determine if small sets of data come from a normal distribution. This involves using the probability properties of the normal distribution. We will eventually make a plot that we hope is linear. We will demonstrate the procedure using the data below.&lt;/p&gt;

&lt;p&gt;Suppose we have ten samples from our process.&lt;/p&gt;

&lt;p&gt;100, 98, 101, 93, 123, 112, 85, 76, 119, 111&lt;/p&gt;

&lt;p&gt;We want to know if we can reasonably assume that these data come from a normal distribution. We can make a normal probability plot to help tell us this.&lt;/p&gt;

&lt;p&gt;The steps below are used to make a normal probability plot of these data.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;1. Sort the data in ascending order.&lt;/em&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;&lt;strong&gt;Data&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Sorted&lt;/strong&gt; &lt;strong&gt;Data&lt;/strong&gt;&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;100&lt;/td&gt;
      &lt;td&gt;76&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;98&lt;/td&gt;
      &lt;td&gt;85&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;101&lt;/td&gt;
      &lt;td&gt;93&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;93&lt;/td&gt;
      &lt;td&gt;98&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;123&lt;/td&gt;
      &lt;td&gt;100&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;112&lt;/td&gt;
      &lt;td&gt;101&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;85&lt;/td&gt;
      &lt;td&gt;111&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;76&lt;/td&gt;
      &lt;td&gt;112&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;119&lt;/td&gt;
      &lt;td&gt;119&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;111&lt;/td&gt;
      &lt;td&gt;123&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;&lt;em&gt;2. Number the sorted data from 1 to n where n is the number of samples (10 in this example).&lt;/em&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;&lt;strong&gt;Data&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Sorted&lt;/strong&gt; &lt;strong&gt;Data&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Number&lt;/strong&gt;&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;100&lt;/td&gt;
      &lt;td&gt;76&lt;/td&gt;
      &lt;td&gt;1&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;98&lt;/td&gt;
      &lt;td&gt;85&lt;/td&gt;
      &lt;td&gt;2&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;101&lt;/td&gt;
      &lt;td&gt;93&lt;/td&gt;
      &lt;td&gt;3&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;93&lt;/td&gt;
      &lt;td&gt;98&lt;/td&gt;
      &lt;td&gt;4&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;123&lt;/td&gt;
      &lt;td&gt;100&lt;/td&gt;
      &lt;td&gt;5&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;112&lt;/td&gt;
      &lt;td&gt;101&lt;/td&gt;
      &lt;td&gt;6&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;85&lt;/td&gt;
      &lt;td&gt;111&lt;/td&gt;
      &lt;td&gt;7&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;76&lt;/td&gt;
      &lt;td&gt;112&lt;/td&gt;
      &lt;td&gt;8&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;119&lt;/td&gt;
      &lt;td&gt;119&lt;/td&gt;
      &lt;td&gt;9&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;111&lt;/td&gt;
      &lt;td&gt;123&lt;/td&gt;
      &lt;td&gt;10&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;&lt;em&gt;3. Calculate (i-0.5)/n for each value; this represents the cumulative probability.&lt;/em&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;&lt;strong&gt;Data&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Sorted&lt;/strong&gt; &lt;strong&gt;Data&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Number&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Cumulative&lt;/strong&gt; &lt;strong&gt;Probability&lt;/strong&gt;&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;100&lt;/td&gt;
      &lt;td&gt;76&lt;/td&gt;
      &lt;td&gt;1&lt;/td&gt;
      &lt;td&gt;0.05&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;98&lt;/td&gt;
      &lt;td&gt;85&lt;/td&gt;
      &lt;td&gt;2&lt;/td&gt;
      &lt;td&gt;0.15&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;101&lt;/td&gt;
      &lt;td&gt;93&lt;/td&gt;
      &lt;td&gt;3&lt;/td&gt;
      &lt;td&gt;0.25&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;93&lt;/td&gt;
      &lt;td&gt;98&lt;/td&gt;
      &lt;td&gt;4&lt;/td&gt;
      &lt;td&gt;0.35&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;123&lt;/td&gt;
      &lt;td&gt;100&lt;/td&gt;
      &lt;td&gt;5&lt;/td&gt;
      &lt;td&gt;0.45&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;112&lt;/td&gt;
      &lt;td&gt;101&lt;/td&gt;
      &lt;td&gt;6&lt;/td&gt;
      &lt;td&gt;0.55&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;85&lt;/td&gt;
      &lt;td&gt;111&lt;/td&gt;
      &lt;td&gt;7&lt;/td&gt;
      &lt;td&gt;0.65&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;76&lt;/td&gt;
      &lt;td&gt;112&lt;/td&gt;
      &lt;td&gt;8&lt;/td&gt;
      &lt;td&gt;0.75&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;119&lt;/td&gt;
      &lt;td&gt;119&lt;/td&gt;
      &lt;td&gt;9&lt;/td&gt;
      &lt;td&gt;0.85&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;111&lt;/td&gt;
      &lt;td&gt;123&lt;/td&gt;
      &lt;td&gt;10&lt;/td&gt;
      &lt;td&gt;0.95&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;&lt;em&gt;4. Determine the z value from the standard normal distribution for each cumulative probability.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;There are a number of ways to do this. The first cumulative probability value is 0.05. You can use the standard normal distribution table in last month’s newsletter to find the value of z corresponding to 0.05 probability. If you look at the table, you will see that z = -1.64 gives a cumulative probability of 0.0505 and a z = -1.65 gives a cumulative probability of 0.0495. So, the value of z that gives a cumulative probability of 0.05 is between -1.65 and -1.64.&lt;/p&gt;

&lt;p&gt;The easiest way to do this is to use Excel’s NORMSINV function. For example, NORMSINV(0.05) = -1.64485. The rest of the values are shown in the table below.&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;&lt;strong&gt;Data&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Sorted&lt;/strong&gt;  &lt;strong&gt;Data&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Number&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;Cumulative&lt;/strong&gt; &lt;strong&gt;Probability&lt;/strong&gt;&lt;/th&gt;
      &lt;th&gt;&lt;strong&gt;z Value&lt;/strong&gt;&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;100&lt;/td&gt;
      &lt;td&gt;76&lt;/td&gt;
      &lt;td&gt;1&lt;/td&gt;
      &lt;td&gt;0.05&lt;/td&gt;
      &lt;td&gt;-1.64485&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;98&lt;/td&gt;
      &lt;td&gt;85&lt;/td&gt;
      &lt;td&gt;2&lt;/td&gt;
      &lt;td&gt;0.15&lt;/td&gt;
      &lt;td&gt;-1.03643&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;101&lt;/td&gt;
      &lt;td&gt;93&lt;/td&gt;
      &lt;td&gt;3&lt;/td&gt;
      &lt;td&gt;0.25&lt;/td&gt;
      &lt;td&gt;-0.67449&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;93&lt;/td&gt;
      &lt;td&gt;98&lt;/td&gt;
      &lt;td&gt;4&lt;/td&gt;
      &lt;td&gt;0.35&lt;/td&gt;
      &lt;td&gt;-0.38532&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;123&lt;/td&gt;
      &lt;td&gt;100&lt;/td&gt;
      &lt;td&gt;5&lt;/td&gt;
      &lt;td&gt;0.45&lt;/td&gt;
      &lt;td&gt;-0.12566&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;112&lt;/td&gt;
      &lt;td&gt;101&lt;/td&gt;
      &lt;td&gt;6&lt;/td&gt;
      &lt;td&gt;0.55&lt;/td&gt;
      &lt;td&gt;0.125661&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;85&lt;/td&gt;
      &lt;td&gt;111&lt;/td&gt;
      &lt;td&gt;7&lt;/td&gt;
      &lt;td&gt;0.65&lt;/td&gt;
      &lt;td&gt;0.385321&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;76&lt;/td&gt;
      &lt;td&gt;112&lt;/td&gt;
      &lt;td&gt;8&lt;/td&gt;
      &lt;td&gt;0.75&lt;/td&gt;
      &lt;td&gt;0.67449&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;119&lt;/td&gt;
      &lt;td&gt;119&lt;/td&gt;
      &lt;td&gt;9&lt;/td&gt;
      &lt;td&gt;0.85&lt;/td&gt;
      &lt;td&gt;1.036433&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;111&lt;/td&gt;
      &lt;td&gt;123&lt;/td&gt;
      &lt;td&gt;10&lt;/td&gt;
      &lt;td&gt;0.95&lt;/td&gt;
      &lt;td&gt;1.644853&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;&lt;em&gt;5. Plot the sorted data versus the z values. The plot is shown below.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/normprobplotnoline.png&quot; alt=&quot;normal probability plot&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The question you want to ask yourself is “Do the points fall roughly in a straight line?” If they do, you can assume that you have a normal distribution. You can see from the chart above, the points appear to fall along a straight line.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/normprobplotwithline.png&quot; alt=&quot;normal probability plot with line&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Since the data fall in a straight line, you can assume that you have a normal distribution.&lt;/p&gt;

&lt;p&gt;If the data do not fall in a straight line, then you cannot assume that you have a normal distribution. The normal probability plot for the non-normal histogram is shown below. Note that it tails like an S at one end. This is often typical of distributions that are not normal.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/normprobplotnonnormaldata.png&quot; alt=&quot;normal prob plot with non-normal data&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;summary&quot;&gt;SUMMARY&lt;/h3&gt;

&lt;p&gt;There are two simple methods of determining if your data are normally distributed. If you have lots of data (100 points or more), you can use a histogram. If the histogram is somewhat bell-shaped, you can assume that you have a normal distribution. If you don’t have lots of data, construct a normal probability plot and see if the points fall roughly in a straight line. If they do, you can assume that your data are normally distributed.&lt;/p&gt;

&lt;hr /&gt;
&lt;h1 id=&quot;5-anderson-darling-test-for-normality&quot;&gt;5. Anderson-Darling Test for Normality&lt;/h1&gt;

&lt;p&gt;You have a set of data. You would like to know if it fits a certain distribution - for example, the normal distribution. Maybe there are a number of statistical tests you want to apply to the data but those tests assume your data are normally distributed? How can you determine if the data are normally distributed. You can construct a histogram and see if it looks like a normal distribution. You could also make a normal probability plot and see if the data falls in a straight line.  We have past newsletters on histograms and making a normal probability plot.  There is an additional test you can apply. It is called the Anderson-Darling test and is the subject of this month’s newsletter.&lt;/p&gt;

&lt;p&gt;We have included an Excel workbook that you can download to perform the Anderson-Darling test for up to 200 data points.  It includes a normal probability plot.  We have also included a link to VBA function macro that you can use to calculate the Anderson-Darling statistic and associated p-value.&lt;/p&gt;

&lt;h3 id=&quot;the-anderson-darling-test-hypotheses&quot;&gt;THE ANDERSON-DARLING TEST HYPOTHESES&lt;/h3&gt;

&lt;p&gt;The Anderson-Darling Test was developed in 1952 by Theodore Anderson and Donald Darling. It is a statistical test of whether or not a dataset comes from a certain probability distribution, e.g., the normal distribution. The test involves calculating the Anderson-Darling statistic.  You can use the Anderson-Darling statistic to compare how well a data set fits different distributions.&lt;/p&gt;

&lt;p&gt;The two hypotheses for the Anderson-Darling test for the normal distribution are given below:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;H0: The data follows the normal distribution&lt;/p&gt;

  &lt;p&gt;H1: The data do not follow the normal distribution&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The null hypothesis is that the data are normally distributed; the alternative hypothesis is that the data are non-normal.&lt;/p&gt;

&lt;p&gt;In many cases (but not all), you can determine a p value for the Anderson-Darling statistic and use that value to help you determine if the test is significant are not. Remember the p (“probability”) value is the probability of getting a result that is more extreme if the null hypothesis is true. If the p value is low (e.g., &amp;lt;=0.05), you conclude that the data do not follow the normal distribution. Remember that you chose the significance level even though many people just use 0.05 the vast majority of the time. We will look at two different data sets and apply the Anderson-Darling test to both sets.&lt;/p&gt;

&lt;h3 id=&quot;two-data-sets&quot;&gt;TWO DATA SETS&lt;/h3&gt;

&lt;p&gt;The first data set comes from Mater Mother’s Hospital in Brisbane, Australia. The data set contains the birth weight, gender, and time of birth of 44 babies born in the 24-hour period of 18 December 1997. The data were explained using four different distributions. We will focus on using the normal distribution, which was applied to the birth weights. The data are shown in the table below.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table of Birth Weights (Grams)&lt;/strong&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;3837&lt;/th&gt;
      &lt;th&gt;3480&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;3334&lt;/td&gt;
      &lt;td&gt;3116&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3554&lt;/td&gt;
      &lt;td&gt;3428&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3838&lt;/td&gt;
      &lt;td&gt;3783&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3625&lt;/td&gt;
      &lt;td&gt;3345&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2208&lt;/td&gt;
      &lt;td&gt;3034&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;1745&lt;/td&gt;
      &lt;td&gt;2184&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2846&lt;/td&gt;
      &lt;td&gt;3300&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3166&lt;/td&gt;
      &lt;td&gt;2383&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3520&lt;/td&gt;
      &lt;td&gt;3428&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3380&lt;/td&gt;
      &lt;td&gt;4162&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3294&lt;/td&gt;
      &lt;td&gt;3630&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2576&lt;/td&gt;
      &lt;td&gt;3406&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3208&lt;/td&gt;
      &lt;td&gt;3402&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3521&lt;/td&gt;
      &lt;td&gt;3500&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3746&lt;/td&gt;
      &lt;td&gt;3736&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3523&lt;/td&gt;
      &lt;td&gt;3370&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2902&lt;/td&gt;
      &lt;td&gt;2121&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;2635&lt;/td&gt;
      &lt;td&gt;3150&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3920&lt;/td&gt;
      &lt;td&gt;3866&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3690&lt;/td&gt;
      &lt;td&gt;3542&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;3430&lt;/td&gt;
      &lt;td&gt;3278&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;The second set of data involves measuring the lengths of forearms in adult males. The 140 data values are in inches. The data is given in the table below.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table of Forearm Lengths&lt;/strong&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;17.3&lt;/th&gt;
      &lt;th&gt;20.9&lt;/th&gt;
      &lt;th&gt;18.7&lt;/th&gt;
      &lt;th&gt;17.9&lt;/th&gt;
      &lt;th&gt;18.3&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;19&lt;/td&gt;
      &lt;td&gt;18.1&lt;/td&gt;
      &lt;td&gt;18.8&lt;/td&gt;
      &lt;td&gt;19.1&lt;/td&gt;
      &lt;td&gt;17.9&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.2&lt;/td&gt;
      &lt;td&gt;19.4&lt;/td&gt;
      &lt;td&gt;19.4&lt;/td&gt;
      &lt;td&gt;17.3&lt;/td&gt;
      &lt;td&gt;18.3&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;19&lt;/td&gt;
      &lt;td&gt;20.5&lt;/td&gt;
      &lt;td&gt;18.5&lt;/td&gt;
      &lt;td&gt;19.4&lt;/td&gt;
      &lt;td&gt;19.6&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;19&lt;/td&gt;
      &lt;td&gt;20.4&lt;/td&gt;
      &lt;td&gt;18.6&lt;/td&gt;
      &lt;td&gt;18.3&lt;/td&gt;
      &lt;td&gt;19.6&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;20.4&lt;/td&gt;
      &lt;td&gt;16.1&lt;/td&gt;
      &lt;td&gt;19.6&lt;/td&gt;
      &lt;td&gt;19.3&lt;/td&gt;
      &lt;td&gt;21&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.3&lt;/td&gt;
      &lt;td&gt;18.7&lt;/td&gt;
      &lt;td&gt;18.5&lt;/td&gt;
      &lt;td&gt;17.2&lt;/td&gt;
      &lt;td&gt;18&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;19.9&lt;/td&gt;
      &lt;td&gt;18.8&lt;/td&gt;
      &lt;td&gt;20&lt;/td&gt;
      &lt;td&gt;17.5&lt;/td&gt;
      &lt;td&gt;17.9&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.7&lt;/td&gt;
      &lt;td&gt;17.3&lt;/td&gt;
      &lt;td&gt;17.8&lt;/td&gt;
      &lt;td&gt;19.6&lt;/td&gt;
      &lt;td&gt;18.1&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;20.9&lt;/td&gt;
      &lt;td&gt;18.1&lt;/td&gt;
      &lt;td&gt;19.8&lt;/td&gt;
      &lt;td&gt;17.6&lt;/td&gt;
      &lt;td&gt;19.5&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;17.7&lt;/td&gt;
      &lt;td&gt;19.9&lt;/td&gt;
      &lt;td&gt;16.6&lt;/td&gt;
      &lt;td&gt;20&lt;/td&gt;
      &lt;td&gt;17.1&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;19.1&lt;/td&gt;
      &lt;td&gt;19.6&lt;/td&gt;
      &lt;td&gt;19.4&lt;/td&gt;
      &lt;td&gt;19.9&lt;/td&gt;
      &lt;td&gt;18.9&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;19.7&lt;/td&gt;
      &lt;td&gt;18.4&lt;/td&gt;
      &lt;td&gt;19.3&lt;/td&gt;
      &lt;td&gt;16.9&lt;/td&gt;
      &lt;td&gt;18.5&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.1&lt;/td&gt;
      &lt;td&gt;19.5&lt;/td&gt;
      &lt;td&gt;20.1&lt;/td&gt;
      &lt;td&gt;19.5&lt;/td&gt;
      &lt;td&gt;19.2&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.4&lt;/td&gt;
      &lt;td&gt;16.8&lt;/td&gt;
      &lt;td&gt;20.5&lt;/td&gt;
      &lt;td&gt;20.4&lt;/td&gt;
      &lt;td&gt;20.5&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;17.5&lt;/td&gt;
      &lt;td&gt;17.1&lt;/td&gt;
      &lt;td&gt;20&lt;/td&gt;
      &lt;td&gt;19.1&lt;/td&gt;
      &lt;td&gt;18.3&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.9&lt;/td&gt;
      &lt;td&gt;18.9&lt;/td&gt;
      &lt;td&gt;20.8&lt;/td&gt;
      &lt;td&gt;18.5&lt;/td&gt;
      &lt;td&gt;19.4&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;19&lt;/td&gt;
      &lt;td&gt;19.7&lt;/td&gt;
      &lt;td&gt;17.7&lt;/td&gt;
      &lt;td&gt;18.3&lt;/td&gt;
      &lt;td&gt;21.4&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;20.5&lt;/td&gt;
      &lt;td&gt;19.7&lt;/td&gt;
      &lt;td&gt;19.9&lt;/td&gt;
      &lt;td&gt;19.8&lt;/td&gt;
      &lt;td&gt;19&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;17.3&lt;/td&gt;
      &lt;td&gt;19.2&lt;/td&gt;
      &lt;td&gt;18.8&lt;/td&gt;
      &lt;td&gt;19.1&lt;/td&gt;
      &lt;td&gt;18.6&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.3&lt;/td&gt;
      &lt;td&gt;20.6&lt;/td&gt;
      &lt;td&gt;16.4&lt;/td&gt;
      &lt;td&gt;17.5&lt;/td&gt;
      &lt;td&gt;19.5&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.4&lt;/td&gt;
      &lt;td&gt;20.1&lt;/td&gt;
      &lt;td&gt;18.5&lt;/td&gt;
      &lt;td&gt;18.5&lt;/td&gt;
      &lt;td&gt;17.4&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.6&lt;/td&gt;
      &lt;td&gt;18.8&lt;/td&gt;
      &lt;td&gt;19&lt;/td&gt;
      &lt;td&gt;19.3&lt;/td&gt;
      &lt;td&gt;18.5&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;19.8&lt;/td&gt;
      &lt;td&gt;17.1&lt;/td&gt;
      &lt;td&gt;20.6&lt;/td&gt;
      &lt;td&gt;19.1&lt;/td&gt;
      &lt;td&gt;18.4&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;20.2&lt;/td&gt;
      &lt;td&gt;18.6&lt;/td&gt;
      &lt;td&gt;19.2&lt;/td&gt;
      &lt;td&gt;17.4&lt;/td&gt;
      &lt;td&gt;18.3&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.5&lt;/td&gt;
      &lt;td&gt;18&lt;/td&gt;
      &lt;td&gt;17.1&lt;/td&gt;
      &lt;td&gt;16.3&lt;/td&gt;
      &lt;td&gt;20.7&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18.5&lt;/td&gt;
      &lt;td&gt;18.7&lt;/td&gt;
      &lt;td&gt;16.3&lt;/td&gt;
      &lt;td&gt;18.2&lt;/td&gt;
      &lt;td&gt;19.3&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;18&lt;/td&gt;
      &lt;td&gt;20.3&lt;/td&gt;
      &lt;td&gt;17.2&lt;/td&gt;
      &lt;td&gt;18.8&lt;/td&gt;
      &lt;td&gt;17.7&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;h3 id=&quot;the-anderson-darling-test&quot;&gt;THE ANDERSON-DARLING TEST&lt;/h3&gt;

&lt;p&gt;The Anderson-Darling Test will determine if a data set comes from a specified distribution, in our case, the normal distribution. The test makes use of the cumulative distribution function. The Anderson-Darling statistic is given by the following formula:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/ADeq1.png&quot; alt=&quot;Anderson-Darling Statistic&quot; /&gt;&lt;/p&gt;

&lt;p&gt;where n = sample size, F(X) = cumulative distribution function for the specified distribution and i = the ith sample when the data is sorted in ascending order.  You will often see this statistic called A2.&lt;/p&gt;

&lt;p&gt;To demonstrate the calculation using Microsoft Excel and to introduce the workbook, we will use the first five results from the baby weight data. Those five weights are 3837, 3334, 3554, 3838, and 3625 grams. You definitely want to have more data points than this to determine if your data are normally distributed. We will walk through the steps here.  You can download the Excel workbook which will do this for you automatically here: &lt;a href=&quot;https://www.spcforexcel.com/files/Anderson-Darling-Workbook.xls&quot;&gt;download workbook&lt;/a&gt;. Of course, the Anderson-Darling test is included in the SPC for Excel software.&lt;/p&gt;

&lt;p&gt;The data are placed in column E in the workbook. After entering the data, the workbook determines the average, standard deviation and number of data points present The workbook can handle up to 200 data points.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/AD1A.png&quot; alt=&quot;Workbook output&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The next step is to number the data from 1 to n as shown below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/AD1.png&quot; alt=&quot;Workbook output&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The formula in Cell F2 is “=IF(ISBLANK(E2),”“,1)”. The formula in cell F3 is “=IF(ISBLANK(E3),”“,F2+1)”. The formula in cell F3 is copied down the column.&lt;/p&gt;

&lt;p&gt;To calculate the Anderson-Darling statistic, you need to sort the data in ascending order. This is done in column G using the Excel function SMALL(array, k). This function returns the kth smallest number in the array. The sorted data are placed in column G.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/AD2.png&quot; alt=&quot;Workbook output&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The formula in cell G2 is “=IF(ISBLANK(E2), NA(),SMALL(E$2:E$201,F2))”. This formula is copied down the column.  The NA() is used so that Excel will not plot points with no data.&lt;/p&gt;

&lt;p&gt;Now we are ready to calculate F(Xi). Remember, this is the cumulative distribution function. In Excel, you can determine this using either the NORMDIST or NORMSDIST functions. They both will give the same result. We will use the NORMDIST function. The workbook places these results in column H.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/AD3.png&quot; alt=&quot;Worbook Output&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The formula in cell H2 is “=IF(ISBLANK(E2),”“,NORMDIST(G2, $B$3, $B$4, TRUE))”. This formula is copied down column H. The average is in cell B3; the standard deviation in cell B4. Using “TRUE” returns the cumulative distribution function.&lt;/p&gt;

&lt;p&gt;Take a look again at the Anderson-Darling statistic equation:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/ADeq1.png&quot; alt=&quot;anderson-darling equation&quot; /&gt;&lt;/p&gt;

&lt;p&gt;We have F(Xi). The equation shows we need 1-F(Xn-i+1). It takes two steps to get this in the workbook. First the value of 1- F(Xi) is calculated in column I and then the results are sorted in column J. The results are shown below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/AD4.png&quot; alt=&quot;workbook output&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The formula in cells I2 is “=IF(ISBLANK(E2), “”, 1-H2)” and the formula in cell J2 is “=IF(ISBLANK(E2),”“,SMALL(I$2:I$201,F2)).” These are copied down those two columns.&lt;/p&gt;

&lt;p&gt;We are now ready to calculate the summation portion of the equation. So, define the following for the summation term in the Anderson-Darling equation:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/adsum.png&quot; alt=&quot;anderson-darling summation term&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This result is placed in column K in the workbook.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/AD5.png&quot; alt=&quot;workbook output&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The formula in cell K2 is “=IF(ISBLANK(E2),””,(2&lt;em&gt;F2-1)&lt;/em&gt;(LN(H2)+LN(J2)))”. This formula is copied down the column.&lt;/p&gt;

&lt;p&gt;We are now ready to calculate the Anderson-Darling statistic. This is given by:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/adresult.png&quot; alt=&quot;anderson-darling result&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The value of AD needs to be adjusted for small sample sizes. The adjusted AD value is given by:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/adstar.png&quot; alt=&quot;adjusted anderson-darling equation&quot; /&gt;&lt;/p&gt;

&lt;p&gt;For these 5 data points, AD* = .357. The workbook has the following output in columns A and B:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/AD6.png&quot; alt=&quot;Workbook Output&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The last entry is the p value.  That depends on the value of AD*.&lt;/p&gt;

&lt;h3 id=&quot;the-p-value-for-the-adjusted-anderson-darling-statistic&quot;&gt;THE P VALUE FOR THE ADJUSTED ANDERSON-DARLING STATISTIC&lt;/h3&gt;

&lt;p&gt;The calculation of the p value is not straightforward. The reference most people use is R.B. D’Augostino and M.A. Stephens, Eds., 1986, Goodness-of-Fit Techniques, Marcel Dekker. There are different equations depending on the value of AD*. These are given by:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;If AD&lt;em&gt;=&amp;gt;0.6, then p = exp(1.2937 - 5.709(AD&lt;/em&gt;)+ 0.0186(AD*)2&lt;/li&gt;
  &lt;li&gt;If 0.34 &amp;lt; AD* &amp;lt; .6, then p = exp(0.9177 - 4.279(AD&lt;em&gt;) - 1.38(AD&lt;/em&gt;)2&lt;/li&gt;
  &lt;li&gt;If 0.2 &amp;lt; AD* &amp;lt; 0.34, then p = 1 - exp(-8.318 + 42.796(AD&lt;em&gt;)- 59.938(AD&lt;/em&gt;)2)&lt;/li&gt;
  &lt;li&gt;If AD* &amp;lt;= 0.2, then p = 1 - exp(-13.436 + 101.14(AD&lt;em&gt;)- 223.73(AD&lt;/em&gt;)2)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The workbook (and the SPC for Excel software) uses these equations to determine the p value for the Anderson-Darling statistic.&lt;/p&gt;

&lt;h3 id=&quot;applying-the-anderson-darling-test&quot;&gt;APPLYING THE ANDERSON-DARLING TEST&lt;/h3&gt;

&lt;p&gt;Now let’s apply the test to the two sets of data, starting with the baby weight. The question we are asking is - are the baby weight data normally distributed?” The results for that set of data are given below.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;AD = 1.717
AD* =  1.748
p Value = 0.000179&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The p value is less than 0.05. Since the p value is low, we reject the null hypotheses that the data are from a normal distribution. You can construct a normal probability plot of the data. How to do this is explained in our&lt;a href=&quot;http://www.spcforexcel.com/normal-probability-plots&quot;&gt; June 2009 newsletter&lt;/a&gt;.   The normal probability plot is included in the workbook. If the data comes from a normal distribution, the points should fall in a fairly straight line. You can see that this is not the case for these data and confirms that the data does not come from a normal distribution.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/adbabyweight.png&quot; alt=&quot;baby weight normal probability plot&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Now consider the forearm length data.  Again, we are asking the question - are the data normally distributed?  The results for the elbow lengths&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;AD = 0.237
AD* =  0.238
p Value =  0.782045&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Since the p value is large, we accept the null hypotheses that the data are from a normal distribution. The normal probability plot shown below confirms this.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/adforearmlength.png&quot; alt=&quot;normal probability plot for forearm length&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The workbook contains all you need to do the Anderson-Darling test and to see the normal probability plot.  If you prefer to use VBA code, this &lt;a href=&quot;http://www.excelforum.com/excel-worksheet-functions/781110-anderson-darling-normality-sense-check-please.html&quot;&gt;link&lt;/a&gt; gives you the VBA code for an Anderson-Darling function.   You can copy this code into a workbook and use it to calculate AD, AD* and the p value for the data.&lt;/p&gt;

&lt;h3 id=&quot;summary-1&quot;&gt;SUMMARY&lt;/h3&gt;

&lt;p&gt;The Anderson-Darling test is used to determine if a data set follows a specified distribution.  In this newsletter, we applied this test to the normal distribution.  The test involves calculating the Anderson-Darling statistic and then determining the p value for the statistic.  It is often used with the normal probability plot.&lt;/p&gt;

&lt;hr /&gt;
&lt;h1 id=&quot;hypothesis-testing&quot;&gt;Hypothesis Testing&lt;/h1&gt;
&lt;hr /&gt;
&lt;p&gt;What is a hypothesis?  According to the Merriam-Webster’s on-line dictionary, a hypothesis is an idea or theory that is not proven but that leads to further study or discussion.  There are many examples of hypothesis.  For example, people who get flu shots are less likely to get the flu.  Or just the opposite hypothesis: getting a flu shot makes no difference in whether someone gets the flu.&lt;/p&gt;

&lt;p&gt;So, a hypothesis is just a statement of theory.  It may or may not be true.  A drug company can claim that a new drug is better at decreasing blood pressure.   You may claim that the diet plan you created helps people lose more weight than a nationally known diet plan.  All these things are just statements – just hypotheses.&lt;/p&gt;

&lt;p&gt;The hypothesis is the starting point.  From there, we have to test the hypothesis and reach a decision if the hypothesis is probably true or probably false.  Note the word “probably.”  There is always variation – so there is always a chance for you to make the wrong decision.  This month’s publication takes a look at the five steps involved in conducting a hypothesis test.&lt;/p&gt;

&lt;h3 id=&quot;the-problem&quot;&gt;THE PROBLEM&lt;/h3&gt;

&lt;p&gt;A lean six sigma project team is recommending a change in the coating process you are in charge of to help reduce costs.  The key variable in your process is the thickness of the coating.&lt;/p&gt;

&lt;p&gt;The average coating thickness is 5 mil.  You want to be sure that the coating thickness remains the same before you will approve the process change.&lt;/p&gt;

&lt;p&gt;The team wants to perform a hypothesis test to prove that the average coating thickness will not change.  The team will go through the basic five steps of hypothesis testing:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Formulate the null hypothesis and the alternative hypothesis&lt;/li&gt;
  &lt;li&gt;Determine the significance level&lt;/li&gt;
  &lt;li&gt;Collect the data and calculate the sample statistics&lt;/li&gt;
  &lt;li&gt;Calculate the p value for the hypothesis test&lt;/li&gt;
  &lt;li&gt;Compare the p value to the desired significance level&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The details of the five steps are shown below.  However, before those steps are covered, a review of the standard normal distribution is needed.  This will be required when we do some calculations.&lt;/p&gt;

&lt;h3 id=&quot;a-brief-pause-for-the-standard-normal-distribution&quot;&gt;A BRIEF PAUSE FOR THE STANDARD NORMAL DISTRIBUTION&lt;/h3&gt;

&lt;p&gt;We need to digress a moment here because we will need to make use of a special case of the normal distribution – when the average = 0 and the standard deviation = 1. This special case is called the standard normal distribution and is shown in Figure 1.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 1: Standard Normal Distribution&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/standard_normal_distribution.png&quot; alt=&quot;standard_normal_curve&quot; /&gt;&lt;/p&gt;

&lt;p&gt;For this distribution, the area under the curve from -∞ to +∞ is equal to 1.0. In addition, the area under the curve is proportional to the fraction of measurements that fall in that region. These two facts can used to help determine the fraction of measurements that fall above some value (such as a specification limit), below some value, or between two values.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/hypothesis_histogram.png&quot; alt=&quot;histogram&quot; /&gt;The x axis in Figure 1 has “z” values.   Any normal distribution can be converted to the standard normal distribution by using the following to calculate z:&lt;/p&gt;

&lt;p&gt;​										z=  (x- μ)/σ&lt;/p&gt;

&lt;p&gt;where x is some value, μ is the average, and σ is the standard deviation of the x values.  The value of z (the z score) is simply how many standard deviations a value, x, is from the average.&lt;/p&gt;

&lt;p&gt;For example, suppose x is 1.5 standard deviations below the average.  In this case, z = -1.5.  The area below z = -1.5 is the percentage of x values that are more than 1.5 standard deviations below the average.  For z = -1.5, that area is 6.68% as is shown in Figure 1.   If z = 1.5, then the area above z = 1.5 is the percentage of x values that are more than 1.5 standard deviations above the average.  This area is also 6.68%.&lt;/p&gt;

&lt;p&gt;To find the percentage of data within z = -1.5 and z = 1.5, you simply use the fact that the area under the curve is 100%, so the percentage of data between the two z values is 100 – 6.68 – 6.68 = 86.64%.  You can determine these percentages from a table of z values (see our publication on the &lt;a href=&quot;https://www.spcforexcel.com/knowledge/basic-statistics/normal-distribution&quot;&gt;normal distribution&lt;/a&gt;) or by using Excel’s NORMSDIST function.&lt;/p&gt;

&lt;p&gt;These percentages can also be viewed as probabilities, e.g., the probability of getting a result that is less than -1.5 standard deviations below the average is 0.0668.  We will make use of this knowledge below.  Now back to the steps in hypothesis testing.&lt;/p&gt;

&lt;h3 id=&quot;step-1-formulate-the-null-hypothesis-and-alternative-hypothesis&quot;&gt;STEP 1: FORMULATE THE NULL HYPOTHESIS AND ALTERNATIVE HYPOTHESIS&lt;/h3&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/hypothesis_testing_1.png&quot; alt=&quot;Hypothesis testing&quot; /&gt;You probably have heard of the null hypothesis and alternative hypothesis.  We will start with the null hypothesis, which is denoted by H0.  Remember, you want to investigate if the process change will impact the average coating thickness.  The null hypothesis is set up to assume that nothing changes – that the status quo holds – or in this case, the process change will not impact the average coating thickness.&lt;/p&gt;

&lt;p&gt;So the null hypothesis (H0) is that the process change will not impact the average coating thickness; the average coating thickness (μ) will remain at 5.  This is usually written as:&lt;/p&gt;

&lt;p&gt;​											H0 = 5&lt;/p&gt;

&lt;p&gt;Now for the alternative hypothesis, which is denoted by H1.  The alternative hypothesis is that the process change will have an effect on the average coating thickness and the average coating thickness will not equal 5.  This is usually written as:&lt;/p&gt;

&lt;p&gt;​											H1 ≠ 5&lt;/p&gt;

&lt;p&gt;This is called a two-sided hypothesis test since you are only interested if the mean is not equal to 5.  You can have one-sided tests where you want the mean to be greater than or less than some value.&lt;/p&gt;

&lt;h3 id=&quot;step-2-determine-the-significance-level-you-want&quot;&gt;STEP 2: DETERMINE THE SIGNIFICANCE LEVEL YOU WANT&lt;/h3&gt;

&lt;p&gt;The significance level is important in hypothesis testing.  It is the probability of rejecting the null hypothesis when it is true.  This probability is denoted by α.  Typical values of α include 0.05 and 0.01.  You decide that you want α to be 0.05.  This means that there is only a 5% of chance of rejecting the null hypothesis when it is actually true.&lt;/p&gt;

&lt;h3 id=&quot;step-3-collect-the-data-and-calculate-the-sample-statistics&quot;&gt;STEP 3: COLLECT THE DATA AND CALCULATE THE SAMPLE STATISTICS&lt;/h3&gt;

&lt;p&gt;The process change is made and data are collected.  The team recommended collecting 25 samples over time.  (Note: The choice of sample size is very important.  This will be subject of next month’s publication.)  The average coating thickness was measured for each sample.  The following statistics were then calculated:&lt;/p&gt;

&lt;p&gt;​							X  = average coating thickness = 5.06&lt;/p&gt;

&lt;p&gt;​						s = standard deviation of the coating thickness = 0.20&lt;/p&gt;

&lt;p&gt;We have our statistics.  How do you decide to accept or reject the null hypothesis?  The way you do this is to assume that the null hypothesis is true and then determine the probability (p value) of getting this sample average.  If the p value is large, it means that there is large probability of getting an average thickness of 5.06 with a standard deviation of 0.20 when the null hypothesis is true and you will accept that the null hypothesis is probably true.  But if the probability of getting these statistics is small, you will assume that the null hypothesis is probably not true and reject it in favor the alternative hypothesis.&lt;/p&gt;

&lt;h3 id=&quot;step-4-calculate-the-p-value&quot;&gt;STEP 4: CALCULATE THE P VALUE&lt;/h3&gt;

&lt;p&gt;To determine this probability, you will need to consider your sampling distribution.    The distribution of sample averages tends to be normal when the sample size is large enough.  We will use this assumption here.  So, your sampling distribution is represented by all the possible sample averages of sample size 25 from the population of coating thicknesses.  This normal distribution is shown in Figure 2.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 2: Normal Distribution for Sample Averages&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/sampling_distribution.png&quot; alt=&quot;sampling distribution&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The highest point on the curve is the average.  The population average of the sample averages (μX ) is equal to the population average, μ, so we have just used μ in Figure 1.  The standard deviation of the sample averages is denoted by σX.&lt;/p&gt;

&lt;p&gt;To be able to draw your sampling distribution, you need to know μX  and  σX.  Since you assumed that the null hypothesis is true, μX  = 5.0.  The standard deviation of the sample averages is given by:&lt;/p&gt;

&lt;p&gt;​										σX =σ/√n&lt;/p&gt;

&lt;p&gt;where σ is the population standard deviation and n is the sample size.&lt;/p&gt;

&lt;p&gt;You don’t know what the population standard deviation is, but you have an estimate from the sample statistics.  The standard deviation of the 25 samples was 0.2.  You can use this as the population standard deviation.&lt;/p&gt;

&lt;p&gt;​								σX =σ/√n =  s/√n=0.2/√25=0.04&lt;/p&gt;

&lt;p&gt;Now you can draw the sampling distribution and add the sample average as shown in Figure 3.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figure 3: Sampling Distribution&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://www.spcforexcel.com/files/images/sampling_distribution_5.png&quot; alt=&quot;sampling distrbution with mean = 5&quot; /&gt;&lt;/p&gt;

&lt;p&gt;You want to know what the probability of getting X = 5.06 is with this sampling distribution.  You can view this probability as how far from μ the sample average is.  The further away it is, the smaller the probability of getting X = 5.06 with this sampling distribution.&lt;/p&gt;

&lt;p&gt;Now we return to the z score.  Remember, the z score is a measure of how many standard deviations the sample average (X )is from the population average (μ).   For this example, the z value is calculated as:&lt;/p&gt;

&lt;p&gt;​								z=  (X-μ)/σX =(5.06-5)/.04=.06/.04=1.5&lt;/p&gt;

&lt;p&gt;So, 5.06 is 1.5 standard deviations away from the average.    As shown above, the probability of getting a result that is 1.5 standard deviations away from the average is 0.0668.  Remember, this a two-side test, so you didn’t care if the difference was above or below the average.  So, the probability of getting an average that is more than 1.5 standard deviations away from the average is 2(0.0668) = 0.1336 or 13.36%.  This is the p value:&lt;/p&gt;

&lt;p&gt;​										p value = 0.1336&lt;/p&gt;

&lt;p&gt;Remember what the p value represents.  You assumed that the null hypothesis is true.  The p value is the probability of getting this result (or a more extreme result) if the null hypothesis is true.&lt;/p&gt;

&lt;h3 id=&quot;step-5--compare-the-p-value-to-the-desired-significance-level&quot;&gt;STEP 5:  COMPARE THE P VALUE TO THE DESIRED SIGNIFICANCE LEVEL&lt;/h3&gt;

&lt;p&gt;In step 2, we set the significance level at 0.05.  Since our p value is greater than this, we conclude that the coating thickness was not impacted by the process change.  We accept the null hypothesis as probably being true.  If the p value had been less than 0.05, we would rejected the null hypothesis and said that the process change did impact the coating thickness.&lt;/p&gt;

&lt;h3 id=&quot;summary-2&quot;&gt;SUMMARY&lt;/h3&gt;

&lt;p&gt;This newsletter has taken a look at how to perform hypothesis testing.  The five steps are:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Formulate the null hypothesis and the alternative hypothesis&lt;/li&gt;
  &lt;li&gt;Determine the significance level you want&lt;/li&gt;
  &lt;li&gt;Collect the data and calculate the sample statistics&lt;/li&gt;
  &lt;li&gt;Calculate the p value for the hypothesis test&lt;/li&gt;
  &lt;li&gt;Compare the p value to the desired significance level&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The normal distribution was used to demonstrate how hypothesis testing is done.  You will not always be dealing with the normal distribution but the process is essentially the same.  One item that is still to be discussed is how to select the sample size.  This will be the subject of a later publication.&lt;/p&gt;
</description>
        <pubDate>Thu, 31 Jan 2019 00:00:00 +0000</pubDate>
        <link>https://cultivo-hy.github.io/statistics/2019/01/31/Statistics-%EC%A0%95%EB%A6%AC/</link>
        <guid isPermaLink="true">https://cultivo-hy.github.io/statistics/2019/01/31/Statistics-%EC%A0%95%EB%A6%AC/</guid>
        
        <category>statistics</category>
        
        <category>distribution</category>
        
        <category>python</category>
        
        
        <category>statistics</category>
        
      </item>
    
  </channel>
</rss>
