Table of Contents

List of Figures ix

Preface xiii

Introduction xv

Chapter 1 Network Control Based on Smart Communication Paradigm 1

1.1. Motivation 1

1.2. General framework 3

1.3. Main innovations 6

1.3.1. User perception metrics and affective computing 6

1.3.2. Knowledge dissemination 8

1.3.3. Bio-inspired approaches and control theory 9

1.4. Conclusion 10

Chapter 2 Quality of Experience 11

2.1. Motivation 11

2.2. QoE concept 12

2.3. Importance of QoE 14

2.4. QoE metrics 16

2.5. QoE measurement methods 20

2.6. QoS/QoE relationship 23

2.7. Impact of networking on QoE 26

2.7.1. Layered classification of impacts on QoE 26

2.7.2. Impact of user mobility on QoE 28

2.7.3. Impact of network resource utilization and management on QoE 29

2.7.4. Impact of billing and pricing 30

2.8. Conclusion 31

Chapter 3 Content Distribution Network 33

3.1. Motivation 33

3.2. Routing layer 36

3.2.1. Routing in telecommunication network 36

3.2.2. Classical routing algorithms 37

3.2.3. QoS-based routing 38

3.3. Meta-routing layer 42

3.3.1. Server placement 43

3.3.2. Cache organization 45

3.3.3. Server selection 47

3.4. Conclusion 49

Chapter 4 User-driven Routing Algorithm Application for CDN Flow 51

4.1. Introduction 51

4.2. Reinforcement learning and Q-routing 53

4.2.1. Mathematical model of reinforcement learning 56

4.2.2. Value functions 57

4.3. Q-learning 60

4.4. Q-routing 61

4.5. Related works and motivation 62

4.6. QQAR routing algorithm 63

4.6.1. Formal parametric model 64

4.6.2. QQAR algorithm 65

4.6.3. Learning process 68

4.6.4. Simple use case-based example of QQAR 71

4.6.5. Selection process 78

4.7. Experimental results 79

4.7.1. Simulation setup 79

4.7.2. Experimental setup 89

4.7.3. Average MOS score 90

4.7.4. Convergence time 97

4.7.5. Capacity of convergence and fault tolerance 100

4.7.6. Control overheads 102

4.7.7. Packet delivery ratio 103

4.8. Conclusion 104

Chapter 5 User-driven Server Selection Algorithm for CDN Architecture 105

5.1. Introduction 105

5.2. Multi-armed bandit formalization 108

5.2.1. MAB paradigm 108

5.2.2. Applications of MAB 112

5.2.3. Algorithms for MAB 113

5.3. Server selection schemes 119

5.4. Our proposal for QoE-based server selection method 122

5.4.1. Proposed server selection scheme 122

5.4.2. Proposed UCB1-based server selection algorithm 125

5.5. Experimental results 126

5.5.1. Simulation results 126

5.5.2. Real platform results 132

5.6. Acknowledgment 133

5.7. Conclusion 135

Conclusion 137

Bibliography 141

Index 155

About the Author

Abdelhamid Mellouk, UPEC, LiSSi Lab, Paris -Est University, Paris, France.

Hai Anh Tran, UPEC, LiSSi Lab, Paris -Est University, Paris, France.

Said Hoceini, UPEC, LiSSi Lab, Paris -Est University, Paris, France.