使用filesystemwatcher时怎么处理大文件

使用filesystemwatcher时怎么处理大文件

FileSystemWatcher基础
属性：
Path——这个属性告诉FileSystemWatcher它需要监控哪条路径。例如，如果我们将这个属性设为“C:\test”，对象就监控test目录下所有文件发生的所有改变（包括删除，修改，创建，重命名）。
IncludeSubDirectories——这个属性说明FileSystemWatcher对象是否应该监控子目录中（所有文件）发生的改变。
Filter——这个属性允许你过滤掉某些类型的文件发生的变化。例如，如果我们只希望在TXT文件被修改/新建/删除时提交通知，可以将这个属性设为“*txt”。在处理高流量或大型目录时，使用这个属性非常方便。
NotifyFilter——获取或设置要监视的更改类型。可以进一步的过滤要监控的更改类型，如watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
| NotifyFilters.FileName | NotifyFilters.DirectoryName;
事件：
Changed——当被监控的目录中有一个文件被修改时，就提交这个事件。值得注意的是，这个事件可能会被提交多次，即使文件的内容仅仅发生一项改变。这是由于在保存文件时，文件的其它属性也发生了改变。
Created——当被监控的目录新建一个文件时，就提交这个事件。如果你计划用这个事件移动新建的事件，你必须在事件处理器中写入一些错误处理代码，它能处理当前文件被其它进程使用的情况。之所以要这样做，是因为Created事件可能在建立文件的进程释放文件之前就被提交。如果你没有准备正确处理这种情况的代码，就可能出现异常。
Deleted——当被监控的目录中有一个文件被删除，就提交这个事件。
Renamed——当被监控的目录中有一个文件被重命名，就提交这个事件。
注：如果你没有将EnableRaisingEvents设为真，系统不会提交任何一个事件。如果有时FileSystemWatcher对象似乎无法工作，请首先检查EnableRaisingEvents，确保它被设为真。
事件处理
当FileSystemWatcher调用一个事件处理器时，它包含两个自变量——一个叫做“sender”的对象和一个叫做“e”的 FileSystemEventArgs对象。我们感兴趣的自变量为FileSystemEventArgs自变量。这个对象中包含有提交事件的原因。以下是FileSystemEventArgs对象的一些属性：
属性：
Name——这个属性中使事件被提交的文件的名称。其中并不包含文件的路径——只包含使用事件被提交的文件或目录名称。
ChangeType——这是一个WatcherChangeTypes，它指出要提交哪个类型的事件。其有效值包括：
Changed
Created
Deleted
Renamed
FullPath——这个属性中包含使事件被提交的文件的完整路径，包括文件名和目录名。
注意：FileSystemEventArgs对象是监控文件夹下有文件创建、删除、修改时的自变量，如果是重命名的话为RenamedEventArgs对象此时除了FileSystemEventArgs对象的属性值，多了一个OldFullPath，为重命名之前的文件名。

大文件处理
1. Create a new FileSystemWatcher.

1System.IO.FileSystemWatcher fswXmlFileWatcher = new System.IO.FileSystemWatcher(); 2this.fswXmlFileWatcher.EnableRaisingEvents = true; 3this.fswXmlFileWatcher.Path =@"C:\Test" 4//in here I only handle the file created event.5this.fswXmlFileWatcher.Created += newSystem.IO.FileSystemEventHandler(this.fswXmlFileWatcher_Created);

2. Write Code to handle File Created Event.

1private void fswXmlFileWatcher_Created(object sender, System.IO.FileSystemEventArgs e) 2{ 3lock(this) 4{ 5string filePath = e.FullPath; 6//incase the file is huge, it need some time to write the whole. so we wait untill the file is accessable by .Net7while (File.GetAttributes(filePath) == FileAttributes.Offline) 8{ 9Thread.Sleep(500); 10} 11//Ok. We start a new thread to process the file12ImportHelper ih = new ImportHelper(filePath); 13Thread th = new Thread(newThreadStart(ih.ImportXmlFile)); 14th.Start(); 15} 16}

3. what is in the ImportHelper Class.

1public class ImportHelper 2{ 3private string _filePath; 4private long _fileSize; 5private FileInfo _fileInfo; 6 7public ImportHelper(string filePath) 8{ 9this._filePath = filePath; 10this._fileSize = 0; 11this._fileInfo = newFileInfo(this._filePath); 12} 13 14//we need to this method. same reason. cause the file size is huge. 15private bool TestFile() 16{ 17long size = this._fileInfo.Length; 18if( this._fileSize == size ) 19{ 20return true; 21} 22else 23{ 24this._fileSize =size; 25return false; 26} 27} 28 29public void ImportXmlFile() 30{ 31while(!this.TestFile() ) 32{ 33Thread.Sleep(2000); 34} 35//It is ok now. The file is ready for us to process. 36//Write your own function to process the file.37ProcessMyFile(this._filePath) 38} 39}

在一次文本文件变化的时候OnChanged事件会触发两次，这是因为除了文本内容变化之外还有文件其他的属性也变化了例如修改时间。
为了解决这问题，也便于项目当中实际使用，写了下面几个类来实际使用：
主方法：
01 using System;
02 using System.IO;
03
04 namespace test
05 {
06 class Program
07 {
08 static void Main(string[] args)
09 {
10
11
12
13 MyFileSystemWather myWather = new MyFileSystemWather(@"C:\test", "*.txt");
14 myWather.OnChanged += new FileSystemEventHandler(OnChanged);
15 myWather.OnCreated += new FileSystemEventHandler(OnCreated);
16 myWather.OnRenamed += new RenamedEventHandler(OnRenamed);
17 myWather.OnDeleted += new FileSystemEventHandler(OnDeleted);
18 myWather.Start();
19 //由于是控制台程序，加个输入避免主线程执行完毕，看不到监控效果
20 Console.ReadKey();
21
22 }
23
24 private static void OnCreated(object source, FileSystemEventArgs e)
25 {
26
27 Console.WriteLine("文件新建事件处理逻辑");
28
29 }
30
31 private static void OnChanged(object source, FileSystemEventArgs e)
32 {
33
34 Console.WriteLine("文件改变事件处理逻辑");
35 }
36
37 private static void OnDeleted(object source, FileSystemEventArgs e)
38 {
39
40 Console.WriteLine("文件删除事件处理逻辑");
41 }
42
43 private static void OnRenamed(object source, RenamedEventArgs e)
44 {
45
46 Console.WriteLine("文件重命名事件处理逻辑");
47 }
48
49 }
50 }
WatcherProcess类：
01 using System.IO;
02
03 namespace test
04 {
05 public class WatcherProcess
06 {
07 private object sender;
08 private object eParam;
09
10 public event RenamedEventHandler OnRenamed;
11 public event FileSystemEventHandler OnChanged;
12 public event FileSystemEventHandler OnCreated;
13 public event FileSystemEventHandler OnDeleted;
14 public event Completed OnCompleted;
15
16 public WatcherProcess(object sender, object eParam)
17 {
18 this.sender = sender;
19 this.eParam = eParam;
20 }
21
22 public void Process()
23 {
24 if (eParam.GetType() == typeof(RenamedEventArgs))
25 {
26 OnRenamed(sender, (RenamedEventArgs)eParam);
27 OnCompleted(((RenamedEventArgs)eParam).FullPath);
28 }
29 else
30 {
31 FileSystemEventArgs e = (FileSystemEventArgs)eParam;
32 if (e.ChangeType == WatcherChangeTypes.Created)
33 {
34 OnCreated(sender, e);
35 OnCompleted(e.FullPath);
36 }
37 else if (e.ChangeType == WatcherChangeTypes.Changed)
38 {
39 OnChanged(sender, e);
40 OnCompleted(e.FullPath);
41 }
42 else if (e.ChangeType == WatcherChangeTypes.Deleted)
43 {
44 OnDeleted(sender, e);
45 OnCompleted(e.FullPath);
46 }
47 else
48 {
49 OnCompleted(e.FullPath);
50 }
51 }
52 }
53 }
54 }
MyFileSystemWather类：
001 using System;
002 using System.Collections;
003 using System.IO;
004 using System.Threading;
005
006 namespace test
007 {
008
009 public delegate void Completed(string key);
010
011 public class MyFileSystemWather
012 {
013 private FileSystemWatcher fsWather;
014
015 private Hashtable hstbWather;
016
017 public event RenamedEventHandler OnRenamed;
018 public event FileSystemEventHandler OnChanged;
019 public event FileSystemEventHandler OnCreated;
020 public event FileSystemEventHandler OnDeleted;
021
022 ///
023 /// 构造函数
024 ///
025 /// 要监控的路径
026 public MyFileSystemWather(string path, string filter)
027 {
028 if (!Directory.Exists(path))
029 {
030 throw new Exception("找不到路径：" + path);
031 }
032
033 hstbWather = new Hashtable();
034
035 fsWather = new FileSystemWatcher(path);
036 // 是否监控子目录
037 fsWather.IncludeSubdirectories = false;
038 fsWather.Filter = filter;
039 fsWather.Renamed += new RenamedEventHandler(fsWather_Renamed);
040 fsWather.Changed += new FileSystemEventHandler(fsWather_Changed);
041 fsWather.Created += new FileSystemEventHandler(fsWather_Created);
042 fsWather.Deleted += new FileSystemEventHandler(fsWather_Deleted);
043 }
044
045 ///
046 /// 开始监控
047 ///
048 public void Start()
049 {
050 fsWather.EnableRaisingEvents = true;
051 }
052
053 ///
054 /// 停止监控
055 ///
056 public void Stop()
057 {
058 fsWather.EnableRaisingEvents = false;
059 }
060
061 ///
062 /// filesystemWatcher 本身的事件通知处理过程
063 ///
064 ///
065 ///
066 private void fsWather_Renamed(object sender, RenamedEventArgs e)
067 {
068 lock (hstbWather)
069 {
070 hstbWather.Add(e.FullPath, e);
071 }
072
073 WatcherProcess watcherProcess = new WatcherProcess(sender, e);
074 watcherProcess.OnCompleted += new Completed(WatcherProcess_OnCompleted);
075 watcherProcess.OnRenamed += new RenamedEventHandler(WatcherProcess_OnRenamed);
076 Thread thread = new Thread(watcherProcess.Process);
077 thread.Start();
078 }
079
080 private void WatcherProcess_OnRenamed(object sender, RenamedEventArgs e)
081 {
082 OnRenamed(sender, e);
083 }
084
085 private void fsWather_Created(object sender, FileSystemEventArgs e)
086 {
087 lock (hstbWather)
088 {
089 hstbWather.Add(e.FullPath, e);
090 }
091 WatcherProcess watcherProcess = new WatcherProcess(sender, e);
092 watcherProcess.OnCompleted += new Completed(WatcherProcess_OnCompleted);
093 watcherProcess.OnCreated += new FileSystemEventHandler(WatcherProcess_OnCreated);
094 Thread threadDeal = new Thread(watcherProcess.Process);
095 threadDeal.Start();
096 }
097
098 private void WatcherProcess_OnCreated(object sender, FileSystemEventArgs e)
099 {
100 OnCreated(sender, e);
101 }
102
103 private void fsWather_Deleted(object sender, FileSystemEventArgs e)
104 {
105 lock (hstbWather)
106 {
107 hstbWather.Add(e.FullPath, e);
108 }
109 WatcherProcess watcherProcess = new WatcherProcess(sender, e);
110 watcherProcess.OnCompleted += new Completed(WatcherProcess_OnCompleted);
111 watcherProcess.OnDeleted += new FileSystemEventHandler(WatcherProcess_OnDeleted);
112 Thread tdDeal = new Thread(watcherProcess.Process);
113 tdDeal.Start();
114 }
115
116 private void WatcherProcess_OnDeleted(object sender, FileSystemEventArgs e)
117 {
118 OnDeleted(sender, e);
119 }
120
121 private void fsWather_Changed(object sender, FileSystemEventArgs e)
122 {
123 if (e.ChangeType == WatcherChangeTypes.Changed)
124 {
125 if (hstbWather.ContainsKey(e.FullPath))
126 {
127 WatcherChangeTypes oldType = ((FileSystemEventArgs)hstbWather[e.FullPath]).ChangeType;
128 if (oldType == WatcherChangeTypes.Created || oldType == WatcherChangeTypes.Changed)
129 {
130 return;
131 }
132 }
133 }
134
135 lock (hstbWather)
136 {
137 hstbWather.Add(e.FullPath, e);
138 }
139 WatcherProcess watcherProcess = new WatcherProcess(sender, e);
140 watcherProcess.OnCompleted += new Completed(WatcherProcess_OnCompleted);
141 watcherProcess.OnChanged += new FileSystemEventHandler(WatcherProcess_OnChanged);
142 Thread thread = new Thread(watcherProcess.Process);
143 thread.Start();
144 }
145
146 private void WatcherProcess_OnChanged(object sender, FileSystemEventArgs e)
147 {
148 OnChanged(sender, e);
149 }
150
151 public void WatcherProcess_OnCompleted(string key)
152 {
153 lock (hstbWather)
154 {
155 hstbWather.Remove(key);
156 }
157 }
158 }
159 }
使用了线程安全的Hashtable来处理一次改变触发两次事件的问题，要注意的是在实际项目使用中，在通过监控文件事情触发时开一个线程WatcherProcess去处理自己业务逻辑的时候，不管业务逻辑成功或者失败（例如有异常抛出一定要try一下）一定要让WatcherProcess的 Completed也就是MyFileSystemWather的WatcherProcess_OnCompleted执行去移除对应变化文件的Hashtable的key，不然下次此文件改变时是无法触发你的业务逻辑的。
还有就是在进行文件监控的时候， 被监控文件在写入的时候，是会有I/O冲突的，即使写入文件是FileShare.Read的也会出现，要真正解决貌似只有FileMaping方法，但是我的项目中文本的写入软件不是我们能控制的，所以只有用处理异常的方法来解决。