第 1 章 · 第 1 节Chapter 1 · Section 1 基础Core
1.1 数据生成与加载Generating and loading data
能够将 CSV 字符串转换为可计算的字段,并说明合成数据的使用边界。Convert CSV strings to typed fields and explain the limits of synthetic data.
本节要点Key points
- CSV 先按字符串读入,再显式转换数值和日期;字段名映射是维护选择,不是语法要求。CSV values arrive as strings; convert numbers and dates explicitly. Field mapping is a maintenance choice, not a syntax requirement.
- 加载后检查缺失、格式、合理范围和记录数;字段齐全不等于数据准确。After loading, check missing values, formats, plausible ranges and the record count. Complete fields do not guarantee accuracy.
- 合成数据必须标注生成方式和用途边界,不能用来推断真实分布。Synthetic data must state how it was generated and what it may be used for; it cannot support claims about a real population.
01理解概念Understand the concept
案例
本章采用中国石油大学(华东)2025 级新生数据,阐述数据的特性。为了防止数据隐私泄露,数据采用 DeepSeek 大模型生成。数据文件 data/newstudent.csv,含 80 条记录。
AI 合成数据的可信度边界
数据质量四维度(Wang & Strong 1996):准确性 / 完整性 / 一致性 / 时效性。
DeepSeek 生成数据:
- ✅ 完整性:字段齐全
- ⚠️ 准确性:姓名/籍贯是虚构的,不能用作真实人口分析
- ⚠️ 采样偏差:模型倾向"看起来合理"的分布(如民族偏汉族、籍贯偏大城市)
课堂要点:教学场景使用合成数据没问题,但必须标注"合成数据",避免学生误以为真实分布。
加载后的检查顺序
读取 CSV 后,先检查字段是否缺失、格式是否一致、数值是否在合理范围,再做类型转换和分组统计。字段齐全只说明结构完整,不代表数据准确;合成数据还应说明生成方法和用途边界。
Case study
This chapter uses the 2025 new-student data from China University of Petroleum (East China) to illustrate the characteristics of data. To prevent data privacy leaks, the data was generated by the DeepSeek large language model. The data file data/newstudent.csv contains 80 records.
Theory: The Trustworthiness Boundary of AI-Synthesized Data
Four Dimensions of Data Quality (Wang & Strong 1996): Accuracy / Completeness / Consistency / Timeliness.
Data Generated by DeepSeek:
- ✅ Completeness: all fields present
- ⚠️ Accuracy: names/hometowns are fictional and cannot be used for real demographic analysis
- ⚠️ Sampling bias: the model tends toward 'plausible-looking' distributions (e.g., over-representing the Han ethnicity, favoring large cities for hometowns)
Classroom Takeaway: Using synthetic data in teaching is fine, but it must be labeled as 'synthetic data' to prevent students from mistaking it for a real distribution.
What to check after loading
After reading a CSV, check for missing fields, consistent formats and plausible ranges before converting types or aggregating. Complete fields do not guarantee accuracy; synthetic data should also document how it was generated and what it can be used for.
02动手实验Try the experiment
试一试:Try this: 重新加载数据,比较年龄与生日字段的类型。思考:为什么不能直接用这些数据推断真实新生分布?Reload the data and inspect the age and birthday types. Why cannot this dataset describe the real student population?
实验将在进入此区域时加载,需要启用 JavaScript。The experiment loads when you reach this area. JavaScript is required.
Some experiment controls use Chinese labels.
做完后:你应该看到After the experiment: what to look for
03代码与讲解Code and explanation收起 / 展开Collapse / expand +
关键代码 ① 字段名映射
把中文列名映射成统一的英文变量名,方便团队维护、搜索和后续处理。JavaScript 支持中文属性名;使用中文键本身不会造成语法或编码错误。
// 关键点 ①:字段映射 —— 中文 → 英文
const COLUMN_MAP = {
"序号": "id",
"姓名": "name",
"性别": "gender",
"生日": "birthday",
"年龄": "age",
"班级": "class",
};
字段映射的用途:把外部文件的字段名与程序内部约定分开,便于以后更换文件或增加数据源。中文键可以正常使用;是否映射成英文应由项目约定决定。读取、保存文件时统一使用 UTF-8,并检查列名是否含空格。
关键代码 ② 类型转换函数
CSV 加载后所有字段都是字符串,需要把序号/年龄 → Number,生日 → Date 对象。
// 关键点 ②:类型转换函数 (推荐方式)
function transformRow(d) {
return {
id: +d["序号"], // 一元 + 把字符串转 Number
name: d["姓名"].trim(), // trim() 去掉首尾空白
birthday: new Date(d["生日"]), // 字符串 → Date 对象
age: +d["年龄"],
class: d["班级"].trim(),
};
}
类型转换:+d["年龄"] 是把字符串转成数值的简写,也可以使用 Number()。空字符串会被转为 0,应先检查缺失值。日期建议按已知格式使用 d3.timeParse("%Y-%m-%d");trim() 可去掉班级等字段首尾的空格。
关键代码 ③ d3.csv 第二参直接传 row 回调
// 关键点 ③:一行完成加载 + 转换(代码示例 1)
const data = await d3.csv("data/newstudent.csv", transformRow);
// 验证类型确实转换了:
console.log(typeof data[0].age); // "number"
console.log(data[0].birthday instanceof Date); // true
加载过程:d3.csv() 返回 Promise,可用 await 或 .then() 获取结果。row 回调在解析每行时完成转换,省去加载后再写一次 map;它并不是逐字节读取的流式接口。网络请求异步进行,数据解析和回调仍在当前 JavaScript 执行环境中完成。
Key Code ① Field Name Mapping
Map column names to a consistent internal naming convention for maintenance and downstream work. JavaScript supports Unicode property names; Chinese keys are valid.
const COLUMN_MAP = {
"序号": "id",
"姓名": "name",
"性别": "gender",
"生日": "birthday",
"年龄": "age",
"班级": "class",
};
Why map fields?Separate external column names from internal conventions so a data source can change without rewriting every calculation. Chinese keys are valid. Choose a convention, use UTF-8 and check whitespace in headers.
Key Code ② Type Conversion Functions
After CSV loading, all fields are strings; you need to convert the index/age to Number and the birthday to a Date object.
function transformRow(d) {
return {
id: +d["序号"],
name: d["姓名"].trim(),
birthday: new Date(d["生日"]),
age: +d["年龄"],
class: d["班级"].trim(),
};
}
Type conversion:Unary + is a concise numeric conversion, not a universal performance guarantee. Check missing values first because an empty string becomes zero. Parse known date formats explicitly with d3.timeParse and trim text fields.
Key Code ③ Pass the row callback directly as the second argument of d3.csv
const data = await d3.csv("data/newstudent.csv", transformRow);
console.log(typeof data[0].age); // "number"
console.log(data[0].birthday instanceof Date); // true
Loading:d3.csv returns a Promise; use await or .then(). Its row accessor transforms each parsed row, avoiding a separate map step. This is not a streaming byte-by-byte API. Network loading is asynchronous; parsing and accessors still execute in JavaScript.
04练习Quiz
学完后,可以标记进度。Mark this section when you finish.
